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Abstract 

INSTRUCTIONAL PLAN EFFECTIVENESS USING DISTORTED AUDIO 
TRANSCRIPTION DURING EARLY STAGES OF ADULTS 
LEARNING A FOREIGN LANGUAGE 

The purpose of the study was (a) to develop lev el- appropriate distorted audio materials 
for early-stage learners of Hebrew and (b) to evaluate the effectiveness of a proposed 
instructional plan to help such students in the transcription of these distorted materials. 
The study was qualitative; however it involved the use of mixed methodology. The 
qualitative part of the study used students' weekly feedback regarding their experience in 
the task of transcribing audio files. The quantitative part analyzed the transcriptions and 
produced numerical values. Due to the small sample of population, all results are 
qualitative and describe trends. The main results show that (a) the developed auditory 
instructional materials are efficient and, using the suggested methods, it is possible to 
produce an unlimited amount of realistic, authentic-sounding and level-appropriate 
auditory materials for teaching; and (b) the suggested teaching plan used with these 
materials is effective and improves students' proficiency in transcription. A future 
implication of this research is that it is now possible to introduce level-appropriate, 
distorted auditory materials to adult students in the early stages of learning a foreign 
language. This can be readily accomplished — at zero cost — with existing human 
resources using existing tools to adapt current curricular materials and formulate a 
tailored instructional plan applicable to students of all languages. 
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CHAPTER ONE: THE RESEARCH PROBLEM 
Statement of the Problem 

Students and graduates of the Defense Language Institute (DLI), during their 
Basic Program course and on their next service assignments, are required to understand 
and interpret imperfect audio material of different types as an essential part of their jobs. 
Communication distortions and interferences represent common phenomena encountered 
by DLI graduates as they start their next mission assignment; one of the major challenges 
graduates are faced with is insufficient proficiency in their comprehension of low-quality, 
distorted audio material. In order to bring DLI graduates to a higher level of proficiency 
in comprehending imperfect audio by the end of the Basic Course Program, more 
emphasis must be placed on training them to understand distorted material, and more 
teaching materials that involve such distortions should be prepared and used. However, 
teachers are challenged by the low availability of such instructional materials in the first 
phases of language teaching, as most materials are focused on language acquisition and 
are of clear auditory quality. Because acoustic distortions can hamper a listener's ability 
to understand what he or she is hearing — distortions increase the difficulty level of audio 
clips beyond the presented verbiage — teachers tend to postpone exposure to imperfect 
materials until a later phase of the course. 

Background and Need for the Study 
The "Real" World of Operation vs. the "Ideal" World of Training 

When comparing the "real" world of operations with the "ideal" world of training, 
Loukopoulos, Dismukes, and Barshi (2009) draw a sharp distinction between the two 
work environments and demonstrate how the real world of operations is very different 
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from the ideal world of the training classroom and the procedures manual. This can lead 
to discrepancies between the ideal and the real; to performance decrements, errors, and 
inefficiencies; and to events where safety is compromised when the training assumes an 
idealized world while trainees must cope with the real world. Loukopoulos et al. 
conclude that organizations should carefully analyze the real world of their operations 
and then design their training and procedures to represent that real world rather than some 
ideal world. 

Similar to the arguments made by Loukopoulos et al. (2009), the language 
samples presented to DLI students during their learning are idealized, at least in the early 
stages of learning and, in some cases, in the more advanced phases. These materials do 
not represent the real world of language use. The operational world for which these 
students are being prepared will present them with more degraded language input than 
they are using in the classroom. The real world of language will include background 
chatter, noise, and electronic interferences such as radio broadcasts and telephone 
conversations. Thus, to be truly prepared for their operational world, students should be 
trained to understand the target language under the realistic and naturalistic conditions 
they will encounter once they complete their training. 
The DLI Setting 

In order to enhance graduates' comprehension of low-quality audio material, DLI, 
in its capacity as the premier foreign language educator for the Department of Defense, 
invests effort in both understanding the different factors involved in this challenge and 
developing teaching and training material based on such factors. 
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Part of this effort is the FLO Enhancement Program (FEP). B. Cohen (personal 
communication, December 14, 2011), Director of the FLO Enhancement Program at DLI, 
presented, in a review of FEP's activity at the 201 1 Final Learning Objectives (FLO) 
Enhancement Summit, the task of "Minimize the gap between DLI graduates and 
Cryptologic Analysts" as one of the challenges faced by DLI today. Cohen maintained 
that one of the critical aspects of this gap is the ability to "instantly recognize and 
accurately translate imperfect input." She pointed out some factors that contribute to 
making auditory material "imperfect," such as multiple speakers, phone-line distortions, 
and background noise. 

Cohen presented the Enhanced Flow Activity (EFA) model, developed by the 
FLO Enhancement Program, in various languages. Each EFA is a lesson plan, built 
around an authentic, low-quality audio excerpt in a consistent, structured format that 
incorporates specific techniques and methodologies: activation of thematic schemata, 
activation of grammatical schemata, research, and more. These materials are a taste of 
"the real world" for students, suitable for semesters 2 or 3 of the basic course — a higher- 
level challenge due to the authentic nature of most of the clips. In her presentation, 
Cohen stressed that introducing students as early as possible to EFA structure and 
low-level material is one of the present goals of the program. Some EFAs in Hebrew 
were developed by the writer to be used by students in weeks 2-16 in the framework of 
the FEP, in order to familiarize them with the structure of the lesson plan. Teaching 
techniques implemented in EFA instruction, along with other techniques, are listed in 
Chapter 3 under "Instrument Description." 
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The update of the Defense Language Proficiency Tests (DLPT) from version 4 to 
version 5 at the DLI is another of the efforts underway to increase the proficiency of 
graduates. In order to meet this challenge on both theoretical and practical grounds, a 
study was requested by DLI in 2010 from the Center for Advanced Study of Language 
(CASL), which conducted a literature review that summarized the current scientific 
research regarding the factors that make second language (L2) listening difficult. 
This comprehensive report provides a framework to be used for the assessment of 
authentic auditory passages in relation to their impact on second language listening 
comprehension. The CASL (2010) report states, "In most cases, only a handful of studies 
have explored the role of these factors in second language listening," and it demonstrates 
that there are many unexplored areas open for further future research. According to the 
CASL (2010) report, three main types of characteristics are considered to have an effect 
on listening comprehension in a second language: "(a) characteristics of the listener, 
(b) characteristics of the passage, and (c) characteristics of the test-taking conditions" 
(CASL, 2010, pp. i-ii). Among factors related to characteristics of the passage, CASL 
(2010) lists transmission noises, communication systems, and phone lines as common 
interferers, stating that "acoustic distortions can have a profound effect on a listener's 
ability to understand what they are [sic] hearing." Additionally, comprehension degrades 
even more when the listener is trying to understand a non-native language (CASL, 2010, 
p. 54). Introduction of distorted materials in the very early stages of teaching a foreign 
language was not mentioned in the CASL (2010) report; as such, the described 
intervention in this study is unique. 
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Other activities involving the use of authentic, low-quality auditory material for 
the purpose of instruction are conducted on a daily basis at DLI by teachers, test writers, 
and curriculum developers — to provide students with the immersion experience as early 
as possible in the course of training. Many of the teaching techniques used by the 
researcher and collected from other teachers are listed in Chapter 3 under "Instrument 
Description." There is a growing need to apply and implement these techniques; students 
must be presented with material of a gradually increasing proficiency level in terms of 
textual content [see "Interagency Language Roundtable (ILR) scale" in the "Definitions" 
section in this chapter] and of a gradually increasing distortion level in terms of auditory 
quality. The availability of such material is low (Richards, 2006), which drives teachers 
and developers to create semi-authentic, low-level auditory materials tailored to their 
teaching needs. 

Rationale for Early Implementation of Instructional Plan 

CASL (2010) suggested that three important factors impact auditory message 
comprehension: (a) exposure to the language, (b) proficiency, and (c) level of anxiety. 
The rationale for early implementation was that the learner would start developing the 
required listening flexibility while applying metacognitive-related listening techniques 
early on in the learning course, thereby increasing total exposure time throughout the 
course. As a result, it was expected that coping with such auditory quality would become 
second nature to him or her, and that anxiety would be reduced. Before this research, one 
downside of early introduction was that the learner lacked sufficient knowledge of the 
language to fill gaps in understanding. One of the findings of this research was that this 
downside can be compensated by a careful selection of auditory materials that do not 
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pose a linguistic challenge to the learner, thereby helping to prevent demoralization and 
demotivation. Specific instructional techniques designed for high-challenge situations 
can be used by the instructor (see "Instructional methods" in Chapter 3) to help students 
cope with limited knowledge of the language. 

To achieve listening flexibility and the gradual development of proficiency 
beginning with the early stages of learning, a teaching plan of several weeks was 
constructed. Such a long-term teaching plan also allowed for metacognitive processes to 
be involved. 

Metacognitive processes play an important role in any adult-learning 
environment. In the case of an adult who is learning the basics of a new language, the 
content or subject matter is usually lower in terms of intellectual level than his or her 
potential thinking level and, as a result, can be boring. The researcher suggested a 
program that offered the adult learner an opportunity to take ownership of his or her 
learning process in a phase of the course where the learner is seeking to add another layer 
of cognitive interest to the learning process, as O'Malley & Chamot (1995) i Wenden 
(1998), and CASL (2010) explain. The presented plan offers the opportunity to expand 
the listening challenge beyond the word/sentence teaching levels and initial vocabulary 
acquisition that are typical of the first weeks of instruction. Through various teaching 
techniques, students could learn (a) to use grammar as a tool to identify words that they 
miss, (e.g., to be able to expect a noun within a compound or an adjective in relation to a 
noun and (b) to choose which listening technique to apply in a given situation. In 
addition, students were able to understand how pronunciation can drive a word's spelling 
and vice versa. The use of different teaching techniques — beyond being efficient — added 
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a necessary layer of cognitive interest to the typically mind-numbing early learning stage 
of basic, low-level language acquisition. 

Purpose of the Study 
The purpose of the study was (a) to develop imperfect audio materials that are 
appropriate for early-stage learners of Beginning Hebrew and (b) to evaluate the effects 
of the proposed instructional plan on the ability of such students to cope with the 
materials. The plan involved exposing learners at an early stage of instruction to 
distorted audio material developed especially for the study, while applying specific 
teaching techniques to help them cope with the challenge. The details of the 
instructional plan are presented in Chapter 3 under "Instructional Plan." 

Theoretical Foundation 

Challenges to process an auditory message can emerge for various reasons: 

• linguistic complexity (Cohen, 2003); 

• a noisy and complex auditory environment (Schneider, Li & Daneman, 2007), 

• the fact that the listener is a learner of the language (Cohen, 2003; Fraser, Gagne, 
Alepins & Pascale, 2010), 

• the ephemeral nature of auditory input (Vandergrift, 2010), or medical issues where 
the listener suffers from a hearing loss (Hicks & Tharpe, 2002). 

Research on listening assessment has received less attention than other language 
skills, possibly because there are additional processes involved in extracting meaning 
beyond the cognitive process necessary for reading and beyond the knowledge of 
phonology, vocabulary, and syntax (Buck, 2001; Alderson & Banerjee, 2002; 
Vandergrift, 2010; CASL, 2010), all of which make comprehension more complex 
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for the listener. As a result, Brindley (1991) argues, it poses a challenge to the 
researcher to identify and isolate the relevant factors involved in the process. Buck 
(2001) explains that listening comprehension involves real-time processing and that the 
variable nature of auditory input (e.g., vocal stress, intonation, nonlinear expression of 
ideas, redundancy, hesitation, and nonstandard grammar followed by corrections) makes 
it difficult to process. 

A student's success in understanding the auditory message depends on his or her 
level of (a) language proficiency, (b) anxiety, and (c) training. As a learner is able to use 
more advanced listening strategies to reduce anxiety and stress, his results improve 
(CASL, 2010; Vandergrift, 2006; Vandergrift, 2007; Vandergrift & Tafaghodtari, 2010; 
Vandergrift, 2010). Anxiety and stress have a negative impact on both the ability to 
focus on the message and the ability to use metacognitive strategies in listening. 
These works exemplify how such characteristics and features mutually affect the final 
results and how, in order to enhance listening efficiency, all these factors need to be 
addressed simultaneously. 

Research Questions 

RQ: Which teaching materials and teaching techniques appear to be most 
effective in helping learners acquire the ability to cope with low-quality audio? 

Substantive Hypotheses 

When tasked with the transcription of imperfect auditory materials, between the 
middle of the 1 st semester and the middle of the 2 nd semester (starting in weeks 6-8, 
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approximately 16 weeks) 1 , students who experience an intervention of exposure to 
imperfect audio materials will 

1. report that some of the teaching and listening techniques helped them more than 
others to acquire the ability to cope with low-quality audio, 

2. report that their stress level gradually declined during the 16 weeks of instruction, 

3. report that their confidence in the accuracy of their transcriptions gradually improved 
during the 16 weeks of instruction, and 

4. gradually increase their level of transcription accuracy as assessed by the instructor. 

During the research, and as reflected on the data collection sheets, additional 
questions and answers emerged, which added to the collected knowledge and supported 
the previous findings of past research. All phenomena and findings are presented in 
Chapter 4 of this paper. 

Description of Study 

This research consists of piloting an instructional intervention for learners in the 
beginning phases of DLFs Basic Hebrew Program. The intervention introduced distorted 
listening materials to students in early stages of the course: between the end of the 1 st 
semester and the middle of the 2 nd semester (week 13-23, for 1 1 weeks). The materials 
were selected from the current available curricular course material or were prepared by 
teachers in the same manner as is done while preparing regular teaching materials 
throughout the year. The elements of the rationale for early introduction of distorted 
audio as a tool to enhance listening skill were: 

1 In practice, the research was conducted in weeks 13-23, due to approval time length and was 1 1 weeks 
long, because results were received earlier than expected. This note is relevant to all following substantive 
hypotheses. 
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1. Increased practice time. As expressed by Cohen in her 201 1 review of FEP's activity 
(B. Cohen, personal communication, December 14, 201 1), early introduction to 
reduced-level material, similar in nature to real distortions that the students will have 
to deal with in the future, can help them better develop the listening flexibility 
required to cope with authentic distorted materials. This concept of increased 
practice time teamed with early introduction of distorted auditory materials is applied 
on a daily basis by the teachers who develop such materials. One element of the 
rationale for early introduction is that at the beginning of the course everything is new 
to the student; therefore, this is a critical time for him/her to learn what to expect from 
the instructional materials. Early introduction of distorted auditory materials would 
initially set higher standards of difficulty, and the initial assumption in the research 
was that the long-term stress level, after an acclimation time, would be equal to or 
closer to that of students who study using standard curricular, high-level audio 
material during standard instruction. Early introduction also gives the student more 
time over the duration of the course to adapt his/her selective hearing and focus on 
the information sought in listening tasks. 

2. Tailored instructional materials. Another assumption is that students do not need 
additional language just because they are exposed to added distortion, as long as they 
are exposed to materials that are at their Interagency Language Roundtable (ILR) 
level (the scale used by DLI to assess a learner's level, see the "Definitions" section 
in this chapter, as well as the full description in Appendix 1). 

In order to cope with the augmented difficulty that the audio presents, the 
students need increased proficiency (i.e., more experience in selective hearing that 
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will enable them to hear through the distortions and focus on the information they 
want to hear). This proficiency developed throughout the research/instructional 
program. An example of a practical scenario that demonstrates such performance is 
how flight controllers do their job. Flight controllers throughout the world 
communicate in English; and, although in many cases their English is poor in terms 
of ILR level, they perform a responsible, professional task while being exposed to 
multiple accents and distorted audio communication. They do not need a high ILR 
level to perform, because they function within a finite professional vocabulary 
(numbers, directions, names, etc.), using defined, simple, professional syntax and 
grammar rules that correspond to the definitions of lower ILR levels. Hence, flight 
controllers adapt to their work environment and develop the proficiency needed to 
function in this limited work environment despite the auditory difficulty. 
3. Availability of support. Students needed extra support, such as instructional 

techniques, to help them with the extra challenge. This support was available (see 
"Instructional Methods" in Chapter 3). 

In this intervention, students in the first semester (weeks 13-23, for 11 weeks) 
were exposed to materials appropriate to their proficiency level at the time (i.e., at each 
instance of exposure, they were able to understand and employ their knowledge of the 
language, even when it was limited). 

The tailored instruction helped the students cope with the presented distortions. 
The pilot program employed the same existing materials that were in use during the same 
weeks in the existing standard curriculum for the Basic Program, but the materials were 
presented to the students in distorted versions. These audios had background noise added 
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to them or were processed to simulate a phone transmission in the Main Study, or were 
distorted in more complex manners in the Advanced Study. As mentioned, in addition to 
the introduction of distorted materials, this intervention incorporated various top-down, 
bottom-up, and other language-specific, tailored teaching methods that are presented and 
discussed further in this dissertation. 

The unmodified original version of the audio materials was not used by other 
teachers in this specific course. The researcher coordinated this issue with all other 
involved teachers of the course. 

At the end of this intervention (middle of the 2 nd semester, week 22), all the 
students wrote an open feedback, where they were asked to reflect on their experience 
during the intervention. The results are reported in Chapter 4 of this paper. 
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Definitions of Terms 

Babble-noise process: Babble noise sounds like multiple people talking at once, 
as at a party, when listener cannot make sense of the noise or conversations (CASL, 
2010). A babble-noise processed clip is created as babble noise and digitally added to the 
parent clip (see definition) to simulate people talking in the background. For technical 
details, please see "Technical Procedures of Audio Clip Preparation" in Appendix 3. 

Bottom-up (strategy): Information processing and knowledge sorting strategy. 
Ruddell & Unrau (2004) explain that when used in reference to a reading model, a 
bottom-up strategy emphasizes a single direction, part-to-whole processing of a text. In 
this research the term refers to the learner's process, starting from the small scale details 
at the sub-word level of grammatical conjugations and hierarchically going up the 
conceptual level (sentence, principle, concept) until reaching a comprehensive 
understanding of the bigger picture. (See also the definition of "top-down" below). 

Coherency/accuracy of transcriptions: The extent to which the listening is 
accurate and the message coherent in the transcriptions, as evaluated by the researcher. 
For more details, see "Coherency/accuracy — numerical grading of transcriptions" in 
Chapter 3. 

Competing Speech process/distortion: In this study, the Competing Speech (CS) 
addition uses a legible weather forecast in the target language. A competing speech- 
processed clip is created when lower-amplitude speech is digitally added to the parent 
clip (see definition). For technical details, please see "Technical Procedures of Audio 
Clip Preparation" in Appendix 3. 
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"Chopped" distortion: In this study, Chopped distortion was produced when the 
parent clip was processed to sound metallic and chopped at the beginning of each new 
word, as it may sound in digitally-coded communications. See also the "Technical 
Procedures of Audio Clip Preparation" in Appendix 3. 

Defense Language Proficiency Tests (DLPT): The DLPT is a battery of foreign language 
tests used by the DLI and the United States Department of Defense (DoD) to measure 
proficiency, as defined by the Interagency Language Roundtable (ILR) Skill Level 
Descriptions (see definition and "Appendix 1"), at the levels of 0+ through 3 (Defense 
Language Institute [DLI], 2009). The tests are used to assess the general language 
proficiency of native speakers of English in the skills of reading, listening, and speaking a 
specific foreign language. The current version of this test is the DLPT5. 

Distortion (audio): Any change of, or addition to, audio material that may reduce 
auditory quality or reduce listeners' ability to comprehend the auditory message. This 
term is used interchangeably with the term "process." 

Distortion Type, Process Type: Includes all types of distortion used in the 
research, including any change in, or addition to, the original audio material or parent clip 
(see definition). 

ILR scale: According to Case Western Reserve University (2010), the Interagency 
Language Roundtable (ILR) scale is a "set of descriptions of abilities to communicate in 
a language." The ILR scale includes five different skill levels. See "Appendix 1" for 
more detail. 
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Metacognition: Metacognition in this research refers to the learner's awareness 
and acknowledgment of his own thinking or cognitive processes or anything related to 
them (Wenden, 1998). 

"Muffled" distortion: This distortion simulates bad communication resulting in a 
muffled voice. It is generated when the parent clip audio is digitally processed and 
becomes muffled. For technical details, please see "Technical Procedures of Audio Clip 
Preparation" in Appendix 3. 

Parent clip: Any original unprocessed clip of high quality. Each parent clip is 
also used as the basis for a generation of distorted clips. 

Phone-line process: The phone-line-processed clip. A phone-line clip is created 
when the parent clip is digitally processed to simulate a phone-line transmission. For 
technical details, please see "Technical Procedures of Audio Clip Preparation" in 
Appendix 3. 

Pink noise: Pink noise is a continuous signal containing all frequencies of a 
defined bandwidth. Unlike white noise (see definition), which carries a flat amplitude 
and spectral density, pink noise carries spectral frequency in such amplitude that the 
energy per Hz is inversely proportional to the frequency. As a result, in pink noise each 
octave carries an equal amount of energy, whereas in white noise each frequency is equal 
in amplitude but not in energy, due to differences in the number of cycles per second 
associated with each frequency. Spectrum and amplitude analysis show that pink noise 
resembles communication interferences more than white noise, which is why pink noise 
was selected for this research. 
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Pink noise/Static process: Pink noise is added to the parent clip to simulate 
background interference typical of communication systems (static). In this paper, this 
process type will also be referred to as "Static." For technical details, please see 
"Technical Procedures of Audio Clip Preparation" in Appendix 3. 

Process (audio): Any change in, or addition to, audio material that may reduce 
auditory quality or reduce the listener's ability to comprehend the auditory message. 

Signal-to-Noise Ratio (SNR): Signal-to-Noise Ratio compares the level of the 
desired main signal, which carries the information, to the level of the background noise. 
The lower the ratio is, the less intelligible the message will be, because background noise 
will mask the information signal (main and background signals are closer in amplitudes). 

Student experience: In this research, the term refers to (a) student indicators (see 
definition below) and (b) evaluation of the different instructional methods. 

Student indicators: In this research, the term refers to the student's (a) reported 
stress level, (b) reported confidence in the accuracy of his/her transcription, and (c) 
perceived difficulty of the transcription task. 

Teacher's effect: This term refers to cases where different teachers teach two or 
more groups who participate in a study. Despite assumed equivalence in teaching 
materials and lesson plans, different teachers' expertise, personalities, and teaching 
styles can affect student achievements, which might be erroneously attributed to the 
different conditions of the experiment (McCaffrey, Lockwood, Kortez, Thomas, & 
Hamilton, 2004). 



17 

Top-down (strategy): Information processing and knowledge-sorting strategy. In 
this research, this term refers to a learner's understanding, building from the bigger 
picture at the conceptual level — starting with the topic or scenario and then hierarchically 
going down to the details (sentences, words, finer details) until reaching full 
understanding. (Compare Bottom-up definition.) 

Transcription quality: In this research, the term refers to three factors in a 
transcript: the number of errors, the type of errors, and the native reader's ability to 
understand the transcription. 

White noise: White noise is a continuous signal containing all frequencies of a 
defined bandwidth, with flat amplitude and spectral density (i.e., the signal contains equal 
power (energy) within a fixed bandwidth). White noise of infinite bandwidth is 
theoretical. White noise bandwidth is, in practice, limited by both the mechanism of the 
noise generator, the medium through which it is transmitted, and the bandwidth capability 
of the listener. In this research, the bandwidth of white noise is defined as the human 
listening bandwidth: 20Hz to 20KHz. 

Limitations 

The researcher was not able to determine whether his convenience sample of 
participants was representative of the population from which it was drawn; therefore, a 
limitation of the research was the unknown extent to which the participants in the study 
were representative of typical beginning students of Hebrew at the DLL 

Delimitations 

1. The participants were DLI students: adults typically ranging in age from 18 to 35 who 
are United States military service members or government agency employees. 
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2. The participants were students in a long and extensive program. 

3. The participants were students of Hebrew. 

4. The participants were in a certain skill level range on the ILR scale (1 to 1 + ). 

Significance of the Study 

The significance of this study is that it addressed the problematic nature of 
teaching imperfect auditory materials; and, in a practical approach, it dealt with the most 
limiting aspects of teaching such materials. Teachers will now have potentially unlimited 
quantities of distorted audio material, at the appropriate language level for their students, 
along with recommended teaching methods to teach these materials. The impact will be 
that teachers will be able to expose students to distorted audio in the early stages of a 
language class, compared to present practice. These benefits became apparent during the 
course of this research: 

1 . Teachers will have instructional techniques and a technical tool to easily create 
imperfect audio materials tailored to students' needs. 

2. Students will have strategies to gradually cope with the auditory challenge and train 
their ear starting early on during the basic course. 

3. Information regarding the success of the pilot program will be available, indicating 
whether students can learn to cope with imperfect auditory materials early in the 
Basic Course. This will allow stakeholders to decide whether the research can be 
used to lay the groundwork for a larger-scale study that will use the procedures 
developed and refined in this pilot study. 
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CHAPTER TWO: REVIEW OF LITERATURE 
Overview 

DLI graduates have difficulty coping with job-related audio input that has low 
sound quality, which poses a problem on their next mission assignment. Early 
introduction of low-level auditory material is one strategy for coping with the challenge. 
This research was (a) a proposed instructional treatment designed to improve DLI 
students' ability to cope with imperfect audio materials in their early stage of learning 
Hebrew, and (b) an analysis of the effectiveness of this plan's components. 

Theoretical Foundation 

Textual vs. Auditory Message 

The area of text typology focuses on rating texts according to difficulty levels, 
especially for the purpose of teaching language to non-native speakers. There are 
multiple factors that need to be taken into account during the process of difficulty 
assessment; among them are vocabulary, grammatical structures of the text, inference and 
subtleties, social competence references, density of text, rhetorical devices, and schemata 
(Cohen, 2003, p. 83). When considering the difficulty assessment of audio material, 
there are additional factors that are not included in the assessment of textual material. 
In addition to the linguistic complexity that exists in textual material (Cohen, 2003), 
challenges to processing different types of auditory messages can emerge from various 
sources. Some causes are considered to be 

• a noisy and complex auditory environment (Schneider, Li & Daneman, 2007); 
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• that the listener is a learner of the language (Cohen, 2003; Fraser, Gagne, Alepins & 
Pascale, 2010); 

• the ephemeral nature of auditory input (Vandergrift, 2010); and 

• medical issues where the listener suffers from a hearing loss (Hicks, 2002; Fraser et 
al., 2010). 

Auditory Quality and Traits 

Fraser et al. (2010) explain that "Many factors can influence how little or how 
much listening effort may be required in any given situation." Some of these factors are 
part of the technical nature of a recording: reduced audio quality, the effect of 
communication platforms such as phone and radio, and magnetic interferences. Other 
traits are part of speaking patterns: different accents (Wilson & Spaulding, 2010), natural 
false starts, repetitions. Some traits are part of conversational characteristics: different 
voices or voices cutting into each other; changing tones, moods, or speech. Others are 
connected to the immediate environment: background noise, echo, and reverberation 
(Klatte, Lachmann, & Meis, 2010, Schneider et al., 2007, Wilson & Spaulding, 2010, 
CASL, 2010). Every listener is challenged by these elements; but when the listener is a 
language learner, s/he is especially challenged and cannot fully comprehend the 
conveyed message. 

Listening Assessment of Language Learners 

Research on listening assessment has received less attention than research on 
other language skills, possibly because of the fact that there are various additional 
processes involved in extracting meaning from text, beyond the cognitive process 
necessary in reading and beyond the knowledge of phonology, vocabulary and syntax 
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(Buck, 2001; Alderson & Banerjee, 2002; Vandergrift, 2010; CASL, 2010), which makes 
comprehension more complex for the listener. As a result, Brindley (1991) argues, 
it poses a challenge to the researcher to identify and isolate the relevant features involved 
in the process. Buck (2001) explains that listening comprehension involves real-time 
processing and that the variable nature of auditory input — which includes stress, 
intonation, nonlinear expression of ideas, redundancy, hesitation, and nonstandard 
grammar followed by corrections — makes it difficult to process. 

Among other factors, a student's success in understanding the auditory message 
depends on the level of language proficiency, the learner's anxiety, and the learner's 
training; as a learner is able to use more advanced listening strategies, reduce anxiety, and 
employ metacognitive strategies his listening, his results improve (CASL ,2010; 
Vandergrift, 2006; Vandergrift, 2007; Vandergrift & Tafaghodtari, 2010; Vandergrift, 
2010). This work exemplifies how these characteristics and features all affect the final 
results, and how, in order to enhance listening efficiency, all these factors need to be 
addressed simultaneously. 
Metacognition 

Metacognitive processes in adult language learning have been receiving attention 
in every conversation related to learning strategies (Anderson, 2002; O'Malley & 
Chamot, 1995; Vandergrift, Goh, Mareschal, & Tafaghodtari, 2006). The awareness that 
adult learning can be enhanced — by thinking about and understanding the learning 
process as it is occurring — opens another avenue in the practice of teaching in general 
and language teaching in particular. The application of metacognition is described by 
Anderson (2002) as "learners who are metacognitively aware know what to do when they 
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don't know what to do; that is, they have strategies for finding out or figuring out what 
they need to do." Anderson continues and divides metacognition into five main 
subprocesses: (a) preparation for the learning process: understanding what the goal is and 
deciding how to reach it; (b) selection and use of learning strategy or strategies known to 
the student in order to efficiently reach the goal; (c) strategy use monitoring: student 
keeps himself on track; (d) combining a number of strategies in order to approach 
different challenges and maximize efficiency of strategy use; and (e) evaluation of 
learning and success: verification of goal achieved. Anderson suggests that, in order to 
maximize efficiency of learning, teaching should include modeling each part of the 
learning process. He maintains that strong metacognitive skills can differentiate a strong 
learner from a challenged one. 
Listening in Auditory Complex Environment 

When a message arrives in an auditory format, additional information is presented 
to the listener. Common listening environments are filled with various types of 
background noise, and whether this additional information is meaningful, such as 
competing speech, or not meaningful, such as the noise of a fan, it poses an additional 
load on the same auditory periphery, activating similar regions in the brain and creating 
either auditory masking or cognitive masking. The auditory masking is in nature an 
"energetic masking," where only first-line parts of the ear, responsible for processing all 
external audio (such as the Basilar membrane), are activated. The higher-level cognitive 
masking, also referred to as "informational masking," appears when information that is 
processed in higher-level areas of the brain is competing with the auditory message at the 
cognitive level (such as phonetic/semantic brain centers). The result in terms of the 
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listener's comprehension of the main message varies from possible interference with 
decoding single words to a total distraction of the listener from the main speaker, 
depending on the volume of the disturbance, the number of disturbances, and the nature 
of the disturbance (Schneider, Li & Daneman, 2007, Anderson & Kraus, 2010). 
Signal-to-Noise Ratio (SNR) 

The comparison between the level of the main signal and the level of the 
background noise is an energetic-in-nature factor. The closer the main signal and the 
background signal are in their amplitudes, the lower the ratio is and the more acute the 
disturbance is: The background noise will mask more of the information signal and the 
message will be less intelligible. SNR is the main factor that influences the processing of 
the relevant information when the competing factor is an energetic masker. 

When the listener is not a native speaker of the main informational stream to be 
decoded, it creates another difficulty factor. L2-listener decoding decreased while 
exposed to noise more significantly than when exposed to LI. Rosenhouse, Haik, & 
Kishon-Rabin (2006) report that when exposed to noise, native speakers of Arabic 
performed more poorly in Hebrew, which was their L2. 

Spatial separation is another factor that can play a role in energetic masking 
effectiveness. Schneider et al. (2007) explain that when the masking energy arrives from 
a different direction than the information stream, it can contribute to the effort of the 
listener to segregate different auditory streams and create a more legible information 
decoding. However, when dealing with recorded message decoding, background noise 
should be considered an innate factor that cannot be spatially separated. Hence, the 
excess energy adds to the difficulty. Such scenarios can be expected in recordings of 
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room conversations, phone calls, radio and television transmissions — and can be linked 
to such as modulated frequency distortions and static signals caused by the electronic 
circuits. In this study, in an effort to minimize the effect of the lack of spatial 
segregation, a fixed 12db-SNR-difference was set as a starting point, because, as shown 
by Schneider et al. (2010), this ratio shows the least effect of spatial segregation. The 
ratio was re-calibrated to a different value during the research to increase the difficulty 
level. (Appendix 3, Appendix 5.) 
Sensory Processes 

Aydelott & Bates (2004) show the contrast between a quiet environment and a 
loud one: They explain how linguistic information, transmitted via acoustic signals, can 
be disrupted and masked by noise from the surrounding environment. Aydelott & Bates 
further explain how the language comprehension process is challenged by both 
perceptual and attentional demands on the listener. Anderson & Kraus (2010) explain 
that understanding speech mixed with noise is a "complex task involving interplay of 
sensory and cognitive processes." Both research studies show how these processes can 
be clinically identified and measured qualitatively and quantitatively and how different 
types of acoustic distortions create different results. Anderson & Kraus indicate that 
listener responses to these challenges are training-dependent, connected to lifelong 
experience in language use, and that the previous exposure and training of listeners can 
be a factor in faster, more pronounced neural response and brain waveforms (p. 577). 

On the sensory level, in order to distinguish the main informational message or 
target sound from the environmental noise, the listener must create a "vocal signature" 
and "tag" the speaker's voice as the desired focus. Anderson & Kraus (2010) further 
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explain that the listener has to form an "auditory object based on spectro-temporal cues," 
which is recognition of both the basic frequency and the second harmonic of the sound 
that together constitute the vocal pitch. The vocal pitch recognition process, essential for 
voice segregation, involves top-down, lower-level auditory cognitive processes such as 
short-term memory and attention. The process is neural in nature, involving processes of 
frequency recognition in the sub-cortex, and it can be clinically measured as "auditory 
brainstem response to complex sounds (cABR)." This process is age sensitive: School- 
age children showed better cortical response to speech-in-noise stimuli than young adults, 
who in turn showed better response than older adults. Training also plays a vital role in 
the vocal pitch recognition process, as training-induced brainstem plasticity has 
been demonstrated. 

Declarative vs. Procedural Knowledge 

The difference between declarative and procedural knowledge is the difference 
between knowing what something is and actually doing it skillfully (Lohse & Healy, 
2012). A heart surgery is a set of steps that can be learned and declared by an 
inexperienced person as well as by an experienced one; however, performing the surgery 
skillfully may take years of experience, which is the acquisition of procedural knowledge 
(Kellogg, 2003, p 248). When referring to listening skills, training should be done in 
conditions similar to the work environment in which the trainee will have to function. 
Loukopoulos et al. (2009) draw a sharp distinction between the "the ideal" and "the real." 
They demonstrate how the real world of operations is very different from the ideal world 
of the training classroom and the procedures manual and how the discrepancy between 
the ideal and the real lead to performance decrements, errors, and inefficiencies. The 
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conclusion, according to Loukopoulos et al., is that organizations should carefully 
analyze the real world of their operations and train their personnel accordingly. 

Literature Related to the Study 
Listening Comprehension Process 

Vandergrift (2010) describes listening comprehension as an active and complex 
process with many hidden mental elements, wherein the listener uses known vocabulary 
and prior knowledge to create a comprehensive picture of the conveyed information. In 
the process, the listener "must discriminate between sounds, understand vocabulary and 
grammatical structures, interpret stress and intonation, retain and interpret this within the 
immediate as well as the larger socio-cultural context of the utterance" (Rost, 2002). In 
addition to all these simultaneous processes, the listener must overcome background 
noises and other interferences (Vandergrift, 2010; Schneider et al., 2007; Wilson & 
Spaulding, 2010) and bridge gaps stemming from missing words, contextual conflicts, 
and temporal attention losses that occur as a result of the streaming and irreversible 
nature of listening activity. 

According to Vandergrift (2010), two distinct main processes have to be involved 
simultaneously while comprehension is taking place. One is a "top-down" process, 
where the listener builds the big picture in his mind as he recalls schemata, makes 
predictions, builds a scenario, and incorporates prior world knowledge with streaming 
collected data to fine-tune an emerging hypothesis of what is presented to him. The other 
is a "bottom-up" process, where the listener uses linguistic knowledge such as 
vocabulary, sentence structure, nuances, and order of speakers and events for a more 
detailed theory and accurate final message. The two processes occur simultaneously; 
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however, the use of each process varies according to (a) the nature of the task (gist, 
transcription, or answering a specific question); (b) familiarity of the listener with the 
subject matter; and (c) language proficiency (Vandergrift, 2010). 

In comprehending an auditory message, the listener utilizes voice separation and 
background auditory cues to more accurately decipher the message; however, 
background auditory distractions and other auditory characteristics also require greater 
listening effort. An example is an audio recording of a noisy street: Background sounds 
of car engines, horns, and other cues will help the listener understand the scene better; yet 
the same elements make it difficult to understand the message because of overlapping 
voices and distracting sounds (Fraser et al., 2010). 

Aydelott & Bates (2004), Gernsbacher (1996, 1997) and Marslen-Wilson (1993) 
break down the process of comprehending and understanding a disturbed auditory 
message into (a) the simpler process of single words recognition and (b) the more 
advanced process of integrating them into a sentence. The encoding of sensory 
information and the initial mapping of this information onto word-level occurs early and 
quickly. Brainwave responses to an understanding of a word "are observed even when 
the prime stimulus is presented briefly and then masked, such that subjects are not 
consciously aware of the identity of the prime" (Aydelott & Bates, 2004, p. 3-4); the 
conclusion is that activation most likely occurs rapidly and involves few requirements in 
terms of attentional and/or processing resources. Gernsbacher (1996) and Marslen- 
Wilson (1993) argue that the next step, selecting and integrating the lexical items, is more 
costly in terms of resources, as it involves the more advanced process of suppressing 
words that were actively registered in the brain but do not match the semantic context. 



28 

In addition, the process of integration involves combining the unsuppressed valid words 
and semantic information into a sentence (Aydelott & Bates, 2004, p. 4). 

Listening comprehension can be assisted if the process involves metacognition 
(see definition), where the learner is aware of the cognitive process he is going through 
(O'Malley & Chamot, 1995, p. 99; CASL, 2010). Skill and experience play an important 
role in metacognition, and an experienced listener incorporates more metacognitive 
strategies in the process (Wenden, 1998). 

Comprehension of Spoken Language in Complex Auditory Environments 

In a quiet environment, comprehension of an auditory message, such as a speech 
or a part of a conversation, is easy compared to the same task performed in a complex 
auditory environment, such as at a party, where there are competing noises of music 
and/or other conversations, often referred to as "Babble Noise." When the auditory 
message is recorded, additional auditory information might be added or lost for 
technological reasons, as a result of pink noise or white noise, distortions, and disruptions 
(Schneider et al., 2007). While this statement is true for native speakers, CASL (2010) 
adds that, "these difficulties are even more pronounced when the listener is trying to 
understand a non-native language." 

One factor that has clearly had an effect on listening comprehension is Signal-to- 
Noise Ratio (SNR). In a complex environment, the competing energy (volume level) of 
the background noise simply masks the signal of the conversation in focus and creates 
"energetic masking." Rosenhouse et al. (2007) determined that "background noise had a 
greater detrimental effect than speech rate" on Hebrew L2 speakers and that "the physical 
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properties of the degrading conditions," which constitute the SNR, determine the amount 
of degradation. 

Spatial separation of the main auditory stream that carries the desired information 
for comprehension from the random energetic masking is an important factor for 
comprehension. Freyman, Balakrishnan, and Heifer (2001) and Freyman, Balakrishnan, 
and Heifer (2004) report that when the listener is able to spatially segregate the different 
data streams, intelligibility of the main message increases. Freyman et al. (2001) 
determined that the slightest effect on intelligibility was when the SNR was at 12db. 
Spatial segregation is not possible when the distracting noise is part of a recording; 
hence, this is another factor that differentiates a recording from real-life situations 
(CASL, 2010) and adds difficulty. 

Another contributor to potential miscommunication: One auditory stream, such as 
a conversation, may capture some attributes of another auditory stream simultaneously. 
For example, although babble noise that sounds like many people talking at a cocktail 
party is unintelligible, it may overlap the relevant targeted auditory stream (CASL, 2010). 
In that case, the listener cannot properly parse, identify, and locate the source of the 
different auditory signals due to (a) inadequate spectral separation, (b) inadequate 
prosodic or phonological separation, and (c) inadequate spatial separation; all of which 
may lead to comprehension errors (Schneider et al., 2007; CASL, 2010). 

White Noise, Pink Noise, or loud music is meaningless vocal energy that overlaps 
and masks the main signal one wishes to decipher. These nonlinguistic sources decrease 
intelligibility through creation of energetic load on brain areas that are peripheral to 
speech processing areas (Aydelott & Bates, 2004; Kittredge, Davis & Blumstein, 2006). 
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In this case, the masking is purely energetic; Schneider et al. (2007) explain that the 
activated regions in the basilar membrane of the listener's ear are the same regions that 
are activated when the message is meaningful. When the message is not meaningful, the 
ear is basically overloaded and saturated to the extent that it cannot properly process the 
meaningful signal. However, the meaningless audio does not activate the same higher- 
processing centers in the brain that competing speech or babble noise activate. 

Comprehension in a complex environment, even if the excess masking noise is 
not meaningful, will require more process resources; because, as described by 
Gernsbacher (1996) and Marslen- Wilson (1993), the number of suppressed words that 
will not be registered as meaningful in the context of the main message is greater due to 
the energy overload, similar frequencies, and other masking factors. 

When the added signal is meaningful, it creates a higher level of disruption; 
because it interferes with the activity of the phonetic, linguistic, or semantic brain centers, 
which are involved with higher-level processing of the target message (Schneider et al., 
2007). This condition of competing speech, where two or more streams of information 
are hard to separate as they compete for the listener' s attention and concentration, poses a 
greater challenge for the listener, as it creates higher-order cognitive disruption 
(Schneider et al., 2007; Klatte et al., 2010). Moreover, when the task involves attending 
to more than one source of information, as in the case of competing speech, the listener 
might experience an added difficulty in processing the message due to not correctly 
analyzing the scene. Such failure occurs when the speaker is not sure "who said what to 
whom" and may incorrectly interpret the conveyed message. 
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Measuring and Assessing Listening 

Assessing and investigating listening can be performed in different ways. 
Vandergrift (2007) describes two ways to approach listening investigation and suggests 
that each manner can be used to shed light on another aspect. The division suggested is 
investigation of (a) the product of listening and (b) the process of listening. 

One common test that uses a product of listening is the listening comprehension 
assessment. The scores, when compared with a control group, can be used to determine 
how successful an intervention is when applied to an experimental group. The 
intervention can be teaching strategies, test conditions, or any other factor that can 
influence comprehension. Another use can be when listening comprehension scores are 
correlated with scores from other measures, such as other lingual tests or measures of 
anxiety while listening. These correlations are circumstantial, and the clear nature of 
their relations usually requires further investigation. Additionally, a common use of 
comprehension tests is level assessment and assignment of students. Another listening 
product is free recall protocol. This protocol requires two groups listening to the same 
material, which was manipulated in some way. The participants are requested to recall 
all essential elements or correct ideas they could understand and remember. Both 
listening comprehension and free recall assessments are influenced by other inherent 
factors, such as the learner's concentration and memory capacity or the learner's ability 
to understand the questions and write the answers. These tests require careful 
application; and in addition, they do not provide information about the process of 
comprehension and why concentration or understanding is broken (Berne, 2004; Buck, 
1991; Vandergrift, 2007). 
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Assessments that can identify the process are very powerful and can provide 
insight into the learners' cognitive paths. These assessments are useful for teachers who 
try to improve instruction methods and learning. These assessments use questionnaires 
that follow standard assessments; memory and recall testing invoked by selected 
questions and cues, interviews and diaries that provide information about learners' 
perceptions of their new experiences and instructional strategy, and think-aloud methods 
that provide similar verbal information. (Berne, 2004; Goh, 2000; Vandergrift, 2007). 
Teaching Methods and Strategies for Listening 

Bottom-Up and Top-Down strategies 

While listening to a second language auditory message, the listener creates a 
mental representation of the message. Field (2004), Vandergrift (2007), and Rost (2002) 
describe how listeners use both top-down and bottom-up strategies, depending on 
multiple factors: the linguistic level of both the auditory message and the listener (relative 
level), the topic of the message, the background knowledge of the listener, and the 
learner's former experience in conducting such a task, which relates to proficiency in 
listening methods. 

In 2004, Field maintains that the main difference between the two strategies is 

that 

The terms refer not to particular levels of processing but to directions of 
processing. In a "bottom-up" process, small ("lower level") units are 
progressively reshaped into larger ones; in a "top-down" process, larger 
units exercise an influence over the way in which smaller ones are 
perceived. 

Both Vandergrift (2007) and Field (2004) explain that listeners use more of a top- 
down direction when dealing with a known topic and limited vocabulary, as the context 
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compensates for linguistic deficiency. In contrast, in less familiar situations vocabulary 
and use of word-level information are essential to create and list some initial 
representation of the auditory information. Field (2004) raised the question of correlation 
between each strategy and the lingual proficiency of the learner. He presents the notion: 
"Weaker second language learners worry about not understanding each word of the input. 
They focus their attention at word level and this occupies much working memory 
capacity, preventing them from building the words into higher-level meaning." This 
notion lends itself to the conclusion that early learners and learners with insufficient 
vocabulary will start with a bottom-up direction, collect word-level information, and then 
compensate for their lack of information with previously acquired world knowledge 
while applying a top-down direction strategy. Goh (2000) shows that the less 
experienced the learner is, the more he will suffer from low concentration and the bigger 
the segmentation problem will be, because he will not be able to separate words or keep 
from losing them while listening. Both Field and Vandergrift agree that inexperienced 
listeners rely at first on known information such as word level or slightly bigger chunks 
of information and then create a context-based temporary theory to fill in the blanks. 
This tendency to fall back on assumed situations to complete the missing parts is 
sometimes misleading and is a source of interpretation errors typical for inexperienced 
learners. Most researchers agree that combining both strategies is the most efficient way 
to cope with listening, with an advantage to the bottom-up strategy in unknown situations 
and where schemata have not yet been activated. 
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Data Processing 

Hicks (2002) and Fraser et al. (2010), researching the field of speech 
comprehension in students suffering from hearing loss, maintained that effort is "the 
exertion of physical or mental power" and that listening effort is the amount of attention 
needed to understand speech. 

People with hearing loss experience the effects of (a) background noise, (b) 
reduction in signal strength, and (c) other distortions. Certain qualitative and quantitative 
methods are used to measure the hearing effort of people with hearing loss and compare 
it to the effort of people without hearing loss. These same methods are valid for 
measuring the hearing effort required of learners of a foreign language to understand a 
distorted message and compare it to the hearing effort required of language learners to 
hear the same message without distortion. 

Listening is a covert process. The assessment of listening products and listening 
strategies provides information that is seldom called into question. Results may be not 
enough for statistical validity, and reliability questions may arise. Vandergrift (2007) 
suggested that this limitation can be challenged by using multi-method assessments and 
triangulation of data from different sources. In 2001, Malterud demonstrated how 
qualitative and quantitative strategies can be perceived as complementary rather than 
incompatible and claimed that, "Although procedures for textual interpretation differ 
from those of statistical analysis, because of the different type of data used and questions 
to be answered, the underlying principles are much the same." Malterud proposed that 
the overall standards of relevance, validity, and reflexivity be used for qualitative inquiry 
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and that the conclusions drawn from qualitative data be transferrable and be compared 
with quantitative-drawn conclusions. 
Ability vs. Proficiency 

An important question raised in several researches refers to the ability and 
proficiency of the individual learner in first language (LI) processing and the correlation 
to second (L2) language comprehension. Vandergrift et al. (2006) tested two hypotheses 
and evaluated how they can explain learner performance. The Linguistic Threshold 
Hypothesis (LTH) suggests that a learner who performs poorly in LI will also perform 
poorly in L2. The Linguistic Interdependence Hypothesis (LIH) suggests that poor 
performance in L2 is due to poor knowledge of L2. The research showed that both 
criteria affect learner performance. 
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CHAPTER THREE: METHODOLOGY AND DESIGN 

Introduction 

This qualitative research study incorporates (a) qualitative data collected from 
feedback from one group of learners regarding their experience during the task of 
transcribing low-quality auditory materials, and (b) qualitative comparative analysis of 
performance which will seek to find differences in intelligibility, error types, and other 
differentiating emerging trends between the high-quality and low-quality transcriptions 
made by these participants. 

The goal of the research was to evaluate the effectiveness of instructional 
materials generated for the purpose of the instruction as well as to evaluate the 
effectiveness of the instructional methods on the students, who were exposed to distorted 
audio in the early stages of learning Hebrew. The research involved developing distorted 
auditory materials and piloting a proposed instructional program to help students learn to 
cope with these materials. The study reports students' perceptions of the instructional 
program. The report incorporates qualitative data collected from feedback from the 
participants regarding their experience. Presented findings are tentative, due to small 
sample size. 

Research Questions 

RQ: Which teaching materials and teaching techniques appear to be most 
effective in helping learners acquire the ability to cope with low-quality audio? 
Additional questions were investigated in the data collection process: 
• How much stress do students feel when transcribing low-quality audio passages, and 
will they report less stress by the end of the study? 
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• How confident do students feel in the accuracy of their transcriptions, and will they 
feel more confident by the end of the study? 

• How accurate will the student transcriptions be, and will that accuracy increase by the 
end of the study? 

How Research Questions Were Answered 

The research questions were answered by a qualitative analysis of transcriptions 
and feedback from participants regarding their perceptions. Although the student 
transcriptions were scored on the basis of the number of errors and were given numerical 
values, the emphasis in the analysis of the transcriptions was qualitative. Reliability of 
scoring was to be established by setting clear decision rules for scoring (see "Weekly data 
collection and analysis" in this chapter). 

The main purpose of the study was to identify which teaching materials and 
listening techniques appear to help learners acquire the ability to cope with low-quality 
audio. The answers to this question and the secondary research questions above were 
drawn from (a) ongoing student feedback elicited throughout the intervention and in the 
final interviews, (b) transcription accuracy scores, and (c) researcher's observations of 
student performance during the study. 
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Description of Research Design 

Method and Design 

This quasi-experimental study uses mixed methodology. The qualitative part of 
the study uses students' weekly formative and summative reflections regarding their 
experience of transcribing audio files. The quantitative part of the study analyses the 
transcriptions and produces numerical values. This part is still qualitative in essence, 
because it has no statistical validity and is considered a trend in a qualitative research. 
Data Source — Population and Sample 

The population of this study consisted of all DLI Hebrew Basic Program students 
(N = 30-40) enrolled in classes 21501HE00212/3 12/412/1 13/213. The sample (n = 12) 
was comprised of students from one of the classes (class 21501HE00113, which started 
in October 2012) who volunteered to participate in the study. The program is located at 
the Presidio of Monterey, California. Students in the program are adult learners, typically 
ranging in age from 18-35, who are United States military service members or 
government agency employees. 

The selection of these specific students was a convenience sampling, because the 
researcher was the teacher for the class; and, given the design of the study, the class was 
at an appropriate point in the course. The students were assigned to this class by the DLI, 
using the DLFs standard practice of course and language assignments in accordance with 
standard procedures, and were assumed by the researcher to be representative of typical 
students in the Hebrew Basic Program. For more details regarding the extent to which 
the participants were representative of Basic Program students in Hebrew, see the section 
on "Limitations" in Chapter I. 



39 

Research Overall Description 

The participants of the study were 12 DLI students learning Hebrew as a foreign 
language. Students were adult learners, males and females, ages 18-30, who were United 
States military service members. All students, regardless of their rank and future 
assignment, were trained as voice interceptors. 

The class selected to participate in this study included two sections of 5-6 students 
each. Both sections were treated equally: Both were taught the same instructional 
material by the same teaching team (comprised of the researcher and three other 
teachers), with the exception of the transcription material relating to the research. In this 
case, all students were instructed by the researcher only, to avoid the possibility of 
"different teacher effect." 

After each session (one teaching hour long, 1-2 sessions per week, total 18 
sessions), students produced 

1. Transcription: At each session of the study the students were asked to transcribe parts 
of the curricular material with the only difference being the imperfect audio, as it was 
modified to represent the low and distorted sound quality of material they are likely to 
encounter during future job assignments. 

2. Feedback: Students submitted their written feedback regarding their perception and 
experience of each session. All feedbacks were designed to evaluate the effectiveness 
of the instructional plan as well as to receive suggestions from the students regarding 
the application of future instruction. To elicit more information regarding student 
experience at each specific session, at the end of the main study and the advanced 
study (see research structure below), students were asked to complete an additional 
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overall experience feedback — more extensive and comprehensive — about their 
perceptions of the previous sessions and their recommendations for further study. 
This questionnaire was composed of open questions; and it was anonymous to allow 
students complete honesty. 

Research Structure 

The research included three parts: a preliminary pilot study for checking and 
refining procedures, materials, and instruction methods; a main study; and an advanced 
study. The preliminary and main parts were originally planned in the research proposal. 
Their successful implementation — ahead of schedule — yielded sufficient information to 
answer the original research question and generate additional ideas; therefore, the 
advanced part was initiated. Data Collection Sheets (DCSs) were modified to fit the new 
focus of this part, developed from lessons learned and accumulated experience. 

Preliminary pilot research. Following the Institutional Review Board (IRB) 
approval, the researcher conducted a pilot of his instructional materials and methods in an 
advanced class for three instructional hours. The preliminary part was conducted in one 
session (six students) of an advanced class (week 27). These students were asked to sign 
an informed consent form before this activity, and the researcher followed the same 
procedure that was planned for the Main Study intervention. The goal of this part was to 
test the DCSs, the distortion effectiveness, and the overall design of the research. The 
participants of this pilot study transcribed material appropriate for their level, their "then 
point" in the Basic Program. The researcher piloted as many clips and teaching 

2 As Hebrew does not have many students, there is no other class available at the same level as the class in 
which the researcher conducted the study. Hence, the closest comparable level class available was targeted 
for the preliminary pilot. 
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techniques as possible in order to optimize the chances of success for the Main Study. 
Students provided feedback on (a) data collection sheets, (b) distorted audio and (c) 
teaching techniques. This preliminary pilot allowed the researcher to pilot the data 
collection sheets as well as the distorted audio clips, instructional techniques, and work 
procedures (i.e., administering data collection sheets, assigning audio clips to pairs of 
students, working with the media player programs, and collecting feedback). This 
preliminary pilot allowed the researcher to detect any possible flaws of the research 
elements and correct them before beginning actual data collection. Since the goal was to 
check the research procedures and structure, it could have been conducted on another 
sample of the total population whose level of Hebrew differed from the research. 

Main research. The main research was conducted on the above-described 
participants of the study. This part answered the research questions. Results were 
received much faster than expected (3 weeks vs. 16 weeks planned data collection) and 
suggested further study. 

Advanced research. Following students' positive feedback and new ideas that 
emerged from the main research — and as there was ample time to continue the study — an 
advanced phase was initiated, focusing on more realistic exposures. The advanced part 
introduced more authentic and advanced distortions that were developed to resemble 
different electronic distortions common in real world communications. 

Instrument Description 

Audio Material 

Audio material was selected from the standard teaching material used in the Basic 
Program's curriculum. Selected audio clips, used for transcription in this experimental 
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research, were distorted by adding Pink Noise, Babble Noise, and Phone-Line processing 
in the Main Study and by further additions or distortions in the Advanced Study. 

The distorted material was prepared as follows: 
Main study 

• Phone-Line filtering: Audio was digitally processed to simulate a phone line. See 
"Technical Procedures of Audio Clip Preparation" in Appendix 3 for more detail. 

• Pink Noise addition: Audio was digitally processed, and Pink Noise was added to 
simulate electronic communication interferences. Pink Noise simulates common 
communication interferences that create an energetic load on the ear (for more detail, 
see the "Definitions" section in Chapter 1 and "Technical Procedures of Audio Clip 
Preparation" in Appendix 3). 

• Babble Noise addition: Audio was digitally processed, and Babble Noise was added 
to the parent clip (see the "Definitions" section in Chapter 1) to simulate background 
conversations common in noisy work environments. Babble noise sounds like many 
people talking at a cocktail party, although the "babble" is unintelligible (See also 
"Technical Procedures of Audio Clip Preparation" in Appendix 3). 

Advanced study 

• Competing Speech addition: Audio was digitally processed, and competing speech in 
the target language (news broadcast) was added to the parent clip (see the 
"Definitions" section in Chapter 1) to simulate background conversations common in 
noisy work environments. However, contrary to Babble noise, Competing Speech is 
intelligible and competes better for the listener's attention (See also "Technical 
Procedures of Audio Clip Preparation" in Appendix 3). 
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• Muffled distortion: Audio was digitally processed, and the parent clip was muffled 
(see the "Definitions" section in Chapter 1) to simulate bad communication, resulting 
in a muffled output. See also "Technical Procedures of Audio Clip Preparation" in 
Appendix 3. 

• Chopped distortion: Audio was digitally processed, so the parent clip became metal 
sounding and chopped at the beginning of each new word, as may happen in digitally- 
transferred coded communications. See the "Definitions" section in Chapter 1 and 
"Technical Procedures of Audio Clip Preparation" in Appendix 3. 

These distortions are realistic representations of the common interferences encountered 
by voice interceptors in their work. They represent a mixture of static, overlapping 
communications, and byproducts of digital coding and decoding of auditory data. 
The messages sound overlapped by other speech, partially cut-off, and metallic 
or muffled. 

Instructional Methods 
Introduction 

The researcher conducted all instruction. A variety of instructional methods, 
detailed below, were employed to support student learning and the writing of 
transcriptions. All methods and strategies were available to all participants and were 
employed by the researcher according to students' needs and the challenges presented to 
them by the different characteristics of the audio materials they processed. The 
instructional experience validated or invalidated the effectiveness of various proposed 
methods and, in addition, it yielded specialized teaching techniques for use with 
imperfect audio materials. 
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The detailed instructional methods described in this section can be categorized in 
different ways, according to their characteristics. Some methods are better for the earliest 
stages of language acquisition, others are more effective after the student has acquired 
some initial skills, and some are better when the student has had prior experience of 
the language. 

Another classification of strategies relates to language, as some strategies are 
"cross-language" and can be used in more than one language while others are "language- 
specific" (i.e., can be applied to one specific language only). Another possible grouping 
is "top-down" vs. "bottom-up" concepts, relating to the cognitive effort the learner has to 
invest and to the stage of the course: The more advanced the stage, the higher the 
required effort will be and the more top-down strategies will be used. A "big picture" is 
created at the higher levels of the program, a word-level strategy in the earlier learning 
stages. This "top-down" vs. "bottom-up" grouping can also relate to students' specific 
learning styles: for example, analytical vs. global. 

The following is a list of instructional methods that were used separately or 
combined, as part of tailored instruction that was available to all participants in this 
research and utilized as needed. The efficiency of each method is described in both this 
list and Chapter 4: "Results and Data Analysis." 
Commonly used teaching methods 
Metacognitive process and adult-learner concept 

"Metacognition" refers to students' awareness of their own learning process (see 
the "Definitions" section in Chapter 1). In addition to employing teaching strategies, the 
teacher explained to the students the nature of each strategy and the mental processes 
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involved, in order to enhance the effectiveness of the metacognitive process and to 
increase student motivation. This concept was used extensively and effectively for 
all students. 
Vocabulary acquisition 

This method is the most basic in terms of learning phases, as it starts with the first 
learned word. It is a bottom-up strategy in that at first the student learns as much 
vocabulary as possible and then infers context and topic from the words that s/he 
understands. At the beginning of the Basic Course, instructors use word lists and 
dictation to teach vocabulary. At the advanced levels of proficiency, teachers let students 
listen to recordings with a common theme, where the same or similar words are repeated 
frequently. Students continuously build language and add volume to their vocabulary. 
This is essential if a student is to advance to the next proficiency level. The vocabulary 
acquisition method was used throughout the instruction plan as well as during the 
standard curricular teaching. Vocabulary was built continuously, using tools such as the 
Rapid Rote program, word lists, and simple games. During transcription activities in the 
research, familiarity with known words was reinforced. Fewer than 10% of the words 
and names that students were asked to transcribe were unfamiliar. Homework (HW) 
activities reinforced the repetition of learned material. An added value of the teaching 
program was that, along with teaching the meaning of new words, the researcher 
emphasized other related elements, such as pronunciation, word linking (how words 
sound when they run together), word fragmentation (breaking one word into fragments), 
relative frequency of occurrence, and parts of speech (preposition, verb, etc.). 
Vocabulary knowledge and familiarity seemed to be a crucial element in students' 
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transcription quality; most student errors tended to occur when the word was unfamiliar 
or distorted/covered by distortion to the extent of being unrecognized. Methods that 
compensated successfully for lack of vocabulary were "write to the best of your ability" 
and "Gap bridging." 
Gap Bridging 

The Gap-Bridging method uses a student's cognition by relating to the context, 
theme, and overall understanding of the message conveyed. A top-down strategy, 
effective at the sentence level and higher, gap bridging can be used beginning in the early 
learning phases. For example, a simple sentence like "My name is [Name]" activates 
schemata in the student's mind, and s/he will expect a name at the end of such a sentence. 
When the name or another word is not familiar, the student can try to hear and spell it to 
the best of his ability (another method, see below). After students are given support and 
are able to identify the unfamiliar word, or are given the solution by the teacher, the effort 
they invested in the initial listening and in related activities helped them remember the 
new words and learn which spelling options are available or common in the language. 
The gap-bridging method is very powerful at more advanced levels of language, when 
students can infer an unidentified word from context. This method was used extensively 
and was effective for all students. 
Find a synonym 

By its nature, finding a synonym is a bottom-up, word-level approach. However, 
it is more advanced than gap bridging and can be top-down and contextual. For example, 
in processing the sentence "President Obama [announced] yesterday that he will not 
students, using the rest of the words and syntax, expect a verb in the past tense and 
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replace the missing verb with other known verbs, such as "stated" or "promised." In 
addition, the student learns which synonyms usually replace the unidentified word in the 
type of context involved. 

In the "find the synonym" technique, when a student cannot initially identify an 
unfamiliar word, s/he can, in many instances, hear certain elements or sounds typically 
associated with the function of the word in the sentence or the part of speech this 
unfamiliar word represents, such as a verb's conjugation in a particular tense or verb 
endings, and build these sounds into the synonym. For example, "announced," "stated," 
and "promised" in English all carry the [ed] typical of the past tense, assuming the verb is 
regular. When the student identifies a possible synonym, s/he can try to identify the 
actual word spoken by searching the synonym's own synonyms, selecting a word that 
seems to fit what s/he heard and then listening to the word again to see if his/her 
hypothesis is true. This method was found to be helpful in some cases, but students 
tended not to use it in their transcriptions because they tended to find the real word in the 
text rather than a substitute. 
Put-aside words 

When a student does not understand a certain word in a sequence, the problem is 
bottom-up, the solution top-down. The student is taught that, in many cases, the knowing 
of one specific word is not crucial and that s/he should continue with the rest of the 
transcription. The student is instructed to: (a) write down the word or whatever 
fraction/part of it that s/he hears, (b) leave a blank line to indicate the location of the word 
in the transcription or write down a time stamp for the location of the word in the 
recording, (c) put aside the challenging word, and (d) continue. In many cases, s/he will 
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later identify the word from context or when hearing the same word again. The student 
can go back to the word later, if s/he still feels it is important, and then employ various 
strategies to identify it from context, grammar, and/or language-specific characteristics. 
The essential thing for the student is to learn how to overcome a psychological block. 
Also important is to be cognizant of the "get it all NOW" tendency, more common 
among detail-oriented students than among global learners. Awareness of this tendency, 
along with metacognitive awareness and using the "Deferred Listening" method (see 
below), helped many students. It was especially effective for detail-oriented and "Type 
A" students who wanted to get perfect results and tended to stop transcribing the text 
until they were sure they understood a problematic part. Encouraging those students to 
"just put it aside and get back to it, you are not giving up and will get back to it," to foster 
the expectation that this word will reappear in the coming text, helped the students to 
eventually understand many words that otherwise would have been insurmountable 
obstacles. 

Keyword mapping, find the theme 

This is a bottom-up strategy. The student listens to a whole sentence or 
paragraph and understands as many words as s/he can. Then, putting all known words 
together, the student infers a context and topic from the words, which allows him/her to 
create a theme (e.g., traffic accident) and even a general picture of the situation (e.g., a 
bike and a car collided at a junction). For example, when the student hears "stocks," 
"percentage," and "market" on the first listening, s/he can understand the theme and be 
ready to understand more on the second try. Then, the student can expect words like 
"up/down," names of companies or countries, and numbers relating to percentages; 
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accordingly, the student can refine the picture or theory that s/he builds or bridge an 
unfamiliar word with a possible synonym. This technique was useful and impactful, 
especially when combined with the Repeated Listening technique (see below). Students 
with a global learning approach tended to use it naturally, but detail-oriented students had 
to be convinced to use it, as they tended to start the transcription immediately from the 
first word before seeing the bigger picture. 
Methods designed for high-challenge/low-quality audio 

The following methods are used more extensively with the transcription of lower 
quality material or with students who experience greater-than-normal difficulty in the 
transcription exercises designed for the study. However, the methods were available for 
all participants, at all times, according to their needs. The instructor used these methods 
at his discretion, without regard to the quality of the audio, with students who were 
challenged by the audio they were required to process. Some of the methods were used 
before the research started, in different settings, by the researcher or other faculty. Other 
methods were developed by the researcher expressly to address new challenges that 
emerged during the research. 
Listen-read-speak 

This bottom-up method is effective for new learners. While listening to a 
passage, students would read the corresponding text aloud. This connects text with 
sound: It improves speaking, pronunciation, and word recall. This method involves the 
use of multiple language skills (listening, reading, and speaking). In the suggested plan, 
after acquiring some experience of the language, students can repeat the text and 
experience for themselves how words sound in a sentence. By experiencing it first hand 
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and creating the sound by himself/herself, the student can develop the required sensitivity 
to identify in a recording a phenomenon such as the running of two words together 
because of similar sounds. This technique, although offered, was not used much by the 
students. The researcher's assumption is that it would have been more useful for students 
in earlier learning stages than the students in this research. However, this method — 
without the speaking part — was used extensively. See "Listening while Reading" in 
Table 5. 

Employ grammar rules 

This is a bottom-up strategy using the cognition of the adult learner. The 
principle is cross language; details are language specific. This approach is applicable to 
all learning levels and depends on the complexity of the context. It helps students expect 
certain grammatical elements or parts of speech in a certain morphological construction 
or context. A simple example that relates to a beginning learner: The adjective that 
describes a noun (analogy from English: "little" in the phrase "little house") will agree 
with the noun in gender and number. Because "house" in this example is a singular noun, 
and in Hebrew "house" is a masculine noun, the adjective will take the form of masculine 
singular. In many cases it will also be in an expected position (immediately after the 
noun). When a listener recognizes a noun in a stream of speech but does not recognize 
the next word, s/he should know that (a) a word s/he doesn't recognize in a certain 
position in relation to the noun may be an adjective and that (b) this adjective will end in 
a predictable way, depending on the gender and number of the noun. This expectation to 
find a certain form will help the student recognize or transcribe the unfamiliar word. A 
more complex example for a more advanced learner of Hebrew: The various forms of 
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nouns, verbs, or adjectives can imply tense (past/present/future), number 
(singular/plural), gender, or possession (mine/yours). In Hebrew, as in English, nouns 
can form compounds ("the government of America" in English can be compounded to 
"the American government"), and the noun may be replaced by a corresponding adjective 
in the process (e.g., "America" becomes "American"). In the example above, "of is 
dropped and the nouns switch their order. When a student hears "The American . . .," he 
can expect a noun in its base form and can narrow the options in the search for the 
identity of the last word in the phrase. The same principle (without the change in word 
order) applies in Hebrew. 

1. Note the Noun, find the agreement. Hebrew has a noun-driven syntax: in Hebrew 
the adjective and the verb that relate to a noun (e.g., the big boys climb on the green 
mountain) have to agree with the noun. In that example, "big" and "climb" would 
take a different form if "boys" were "boy," "girl," or "girls." By properly identifying 
the noun, the student can expect a specific form of the adjective. 

2. Note the Verb, find the tense/possession. The Hebrew verb is a central part of a 
sentence and provides clues about the form, mode, or mood in other parts of the 
sentence. By its conjugation, the verb agrees with the noun in gender and number (as 
explained above) and expresses the tense of the sentence. In Hebrew as in English, 
the form of the verb can indicate when an action occurs. For example, "the big boys 
climb on the green mountain" indicates present time, while "the big boys climbed on 
the green mountain" indicates past. Students can benefit when they extract as much 
information as they can from what they hear and then apply their understanding of 
context and knowledge of syntax to transcribe accurately. 
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This method was widely used. Participating students were at the appropriate 
stage to use grammatical cues and hints; therefore, this method was applied extensively. 
It was especially evident when future tense was learned as part of the curriculum and 
started to appear in the transcribed audio of the research. The students were aware of 
the new structure, attempting to remember the grammatical rules that were not yet 
second nature to them and to understand the text by extracting the expected structure 
from memory. 

Use of language characteristics 

This is a bottom-up technique that uses the adult learner's cognition and is 
effective for students towards the middle and end of the Basic Program, when the student 
has gained more proficiency in the language. It is language-specific in application; 
however, the concept is cross-language and can be applied according to each language's 
specific characteristics. 

1. Writing characteristics. This is a bottom-up technique. Students often misspell the 
same sound sequences, using alternate spellings like "KA" and "CA" in English; "Y" 
versus "LL" in Spanish, depending on the dialect; and "J" in Spanish, which 
represents an h-like sound. Awareness of the spelling challenges and options can 
help the learner search the dictionary for different spelling alternatives. 
During the research, transcription was checked for accuracy in hearing and sounds, 
not spelling. "K" and "CA" were both accepted, if the sound was indeed there. For 
this reason, the technique was not widely used in the research. 
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Speaking Characteristics 

a. General speaking tendencies: Students learn of elements that may impede 
understanding — false starts, stuttering, repetitions, rewording, etc. 

b. Speed effects : Students learn what can cause a typical merging of words while 
speaking. For example, when a word ends with a certain sound and is 
followed by a word that begins with the same sound, such as in "I'm Mary," 
speakers tend to merge them into an utterance that can sound like a single 
word. Awareness of this tendency can help students speculate as to where to 
separate words. For example, "My name is Sam" can sound to the new 
listener like "My name issam." Awareness of speed effects is cross-language 
and bottom-up for all learning phases and is applicable to authentic materials, 
in which speakers speak more quickly and use colloquial language. 

The understanding and application of speaking characteristics was one of the most 
effective tools used to overcome the common errors of running words together. It 
was widely used by students and it improved their performance. 
2. Pronunciation characteristics: Each language has its specific accent attributes. 
Learning them improved students' awareness of missing sounds and syllables. For 
example, in English, the phoneme III is pronounced in several ways and there are r- 
less dialects. In Spanish there are two Irl phonemes, as in pew ("but") and perro 
("dog"). In Hebrew, there is one III, and it is guttural and subtle. Americans 
studying Hebrew have to learn to listen for the Hebrew Irl in words; if they miss the 
Irl in a syllable that contains one, they may fail to recognize the word. 
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The American /O/, as in "thin," and IXJ as in "tin" are pronounced by Israelis as 
Itl only, because /0/ does not exist in Hebrew. As a result, even when a cognate such 
as "mathematics" or a name containing /0/ in English is pronounced by an Israeli, it is 
hard to recognize, and it takes some skill and proficiency to decipher it. To 
demonstrate, the name "Martha" (note the III and /0/), when spoken in an Israeli 
accent, poses a challenge to the beginner. 

On the other hand, Hebrew, like Spanish, uses a "n" sound, pronounced the 
way a Spanish speaker pronounces the sound represented by the letters "J" or "X" 
(but with even more subtlety). This h-like sound does not exist in American English 
and is therefore a challenge for Americans learning Hebrew. 

Similarly to speaking characteristics, the understanding and awareness of 
pronunciation characteristics was an effective tool used to overcome errors that stem 
from learning to recognize sounds that are not common in English. This method was 
widely used by students and improved their performance. 
Write down the word to the best of your ability 

The researcher developed this method and the HW exercises described below. It 
is beneficial for learners of languages with different alphabets. When using the "write 
down the word" method, students are asked to write down a familiar or unfamiliar word 
to the best of their ability using the new alphabet and the spelling rules they know. The 
activity of trying to hear and write down the word focuses the student on the sounds the 
word is composed of and increases listening skills. It also reinforces the student's 
familiarity with the sound produced by each letter or combination of letters. In addition, 
it helps develop spelling skills (i.e., learning what the visual representation of sounds may 
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be) and hearing skills (i.e., learning what sounds s/he is likely to hear in the context of 
certain syllables). 

The ability to recognize and attribute the correct sound and spelling, especially 
when they are of an unfamiliar alphabet, and construct them on paper, is very useful to 
the students' next job assignment; undoubtedly a military linguist may encounter 
unfamiliar words on a daily basis. If s/he is able to write them down and search the 
dictionary for them or bring them to his/her superior transcribed in such a way that the 
supervisor can recognize them, then his/her work is successful. This technique promotes 
the ability to do so. 

During this reported research, this method was extremely helpful and the results 
show a direct link between exercises that the students were given to promote this ability 
and the transcription results. 

The following exercises were practiced, mainly at home — the following 
transcriptions showed immediate results (Chapter 4). 

1 . Transliteration in Hebrew: Students were tasked to write English words in Hebrew 
letters. Students were asked to write down in Hebrew letters paragraphs that were 
given to them in English. Suggested game in class: students write something in 
English using the target alphabet and the teacher has to understand it. This method 
was developed by the researcher a number of years ago and was used in past courses. 

2. Transcription of above-the-level material: Students were tasked to transcribe a clear 
recording of material that was taken from a more advanced level compared to their 
actual level. There was no frustration, as students knew that the goal was to work 
only on their hearing and their ability to transfer it to paper in the new alphabet. This 
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method was developed by the researcher a number of years ago and was used with 
courses in the past. In this research, students who were in their second week received 
transcription material from week 17. Afterwards, feedback from the first task was 
given. In the second activity, students returned perfect transcripts of totally 
unfamiliar verbiage to them, which (ignoring spelling errors such as replacing "Ka" 
with "Ca") produced perfect sounds. 
Vowel the text using audio recording 

This method was developed by the researcher, to improve the way students 
transcribe and express what they hear. In fact, it helped them to be more clear when 
relaying the message in writing. Although it does not enhance the student's hearing, it 
does promote the next phase: telling the other what you heard, as it helps the native or 
more experienced reader of the transcription know that the learner heard correctly. For 
example, the word "read" can be pronounced in present tense and in past tense; however, 
it is spelled in the same exact way. How would the reader know what the student has 
actually heard? There is no way to know for sure; during the research the researcher 
noted that although the student may have heard correctly, the researcher was not actually 
sure what was heard. This method can bridge the gap. In order to understand the 
"Vowel the Text" method, some introduction is required: in written Hebrew, vowel 
accents are used above and below words, in addition to vowel letters and, of course, 
consonants. For example, the word "njin" has (a) consonants, (b) vowel letters, and (c) 
vowel accents. However, Modern Hebrew is usually written with only consonant and 
vowel letters, while the vowel accents above and below words are not mandatory and are 
normally used at the discretion of the writer to distinguish between similar words (for 



57 

example, "nnn" vs. "njin"). In transcription exercises, students normally write only the 
consonants and vowel letters, as is the practice in Modern Hebrew, without using the 
vowel accents above and below the words. Using an example from Spanish, "Jose" 
would be written "Jose." 

In the "Vowel the Text" method, students were provided with printed 
transcriptions, with only consonants and vowel letters, and were tasked with adding the 
vowel accents above and below the words while listening to the audio. This is a perfect 
method to (a) break the words into their component sounds, (b) connect sound with 
kinesthetic-driven and visual learning — the student has to focus on each sound and 
transfer it to the paper through hand movement and kinesthetic learning skill, and (c) 
visually see the result and master the correct spelling. This exercise helped to promote 
recognition of the word in the next encounter, whether in oral or written form. 

During this reported research, this method was extremely helpful to connect 
sounds from the audio to the transcription. Students were able to fine-tune their 
transcriptions and relay the specific sounds they heard onto paper by adding vowel 
accents. This method was practiced at home, generating immediate results on the 
subsequent transcriptions, as described in Chapter 4: "Results and Data Analysis." 
Transcription puzzle 

This method was developed by the researcher to promote listening in the noisy 
and challenging auditory environment of advanced students as well as in the environment 
of beginning students. This method connects the auditory message with its transcript: 
The learner makes the connection between the high-challenge auditory representation and 
its visual and correct representation as created by the teacher. This involves listening, 
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reading, and identification of text. This method can be utilized with students who lack 
understanding of the Hebrew language, even if they also lack knowledge of sound- 
symbol correspondence in Hebrew. In this case, students can use their native Roman 
alphabet to transcribe Hebrew sounds. This method involves listening only, which makes 
it ideal for the development of selective hearing , especially in the very early stages of 
learning. The students are supposed to listen to a passage, find the appropriate written 
representations of the words (whether in the Hebrew or Roman alphabet) of the spoken 
passage, and then put them in the right order on the SMART Board screen. The teacher 
creates a "bank" made of a complete transcription (not just a "word bank," from which 
only some selected words may be used, but rather a bank of the complete transcription 
from which any or all words may be selected for use). The student has to put the text in 
order, like a puzzle, using the audio as a guide. The "pieces" of the puzzle consist of 
single words, phrases, or even full sentences, according to the desired difficulty level. 

Making a puzzle is easy: In this study, "SMART Board" and "Paint" software 
were used to create the puzzles. The complexity of the puzzle (size and number of 
chunks/pieces of the puzzle and decisions as to where the text is separated and broken 
into "pieces") is determined by the instructor and can be tailored according to difficulty 
level. The task is to reconstruct a disassembled and scrambled transcription while 
listening to the original listening passage. The student can move the "pieces" around on 
the SMART Board, trying various locations, until s/he successfully reconstructs the text. 
Because the student does not have to write the words, s/he is exposed to them in their 
correct form, and if s/he does not understand what they mean, s/he can easily look them 

3 A listener can hear only what s/he would like to hear by using the cognitive process of attention to 
selectively concentrate on one aspect of the environment while ignoring others. 
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up. This method is ideal for training the ear to detect a word or a number of words by 
just listening and then putting them together. If a given word is more challenging than 
others in the listening passage, the instructor can design the puzzle so that the challenging 
word appears "in context," (i.e., accompanied by preceding and following words). To 
increase the challenge, the instructor may decide to reduce the "piece" in size to fewer 
words or to a single word at a point in the listening passage where the sound is unclear. 
This method is good for the first listening of a distorted text, as well as for a second 
listening after a failed attempt, or as follow-up homework. The researcher employs this 
method with students in the beginning phase of the Basic Program (weeks 2-8) and with 
advanced students (week 43), without relation to the research; during this reported 
research, the method was extremely helpful to train students' ears to attune to a certain 
distortion. The best use of this method in the research was when it was applied as HW or 
a "warm-up" drill before the actual transcription of the session, when previous clips from 
previous sessions, distorted in the same way, were used as the puzzle. This method 
connected the sounds as they were heard through the distortion to the actual text. 
A direct link was observed between the exercises students were given to enhance this 
ability and the transcription results (Chapter 4). 
Listen while Reading/Text-Accompanied Audio 

This is a bottom-up/top-down technique that can be used in all learning phases. 
Students use text and read while listening. This technique helps the learner to relate text 
and spelling to the pronounced word; it supports learning through integration of multiple 
skills. This method is useful only if an accurate transcription is available, as reading 
from an erroneous transcription will result in faulty learning. For example, students were 
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given a HW assignment to listen to a recording while reading a transcription given to 
them by the instructor. This method was useful for low-quality audio and was applied 
when students were initially unable to understand distorted words. Using this method, 
students were able to acquire listening proficiency without frustration. They trained their 
brain to focus on expected sounds and ignore distortions; having read the text, they 
expected the words and sought them in the audio. This approach is most effective if the 
students first listen without the transcript until they reach the point where, for some 
portion or portions of a passage, they feel "stuck" (i.e., unable to understand anything 
more even after repeated listening attempts). If they then consult the transcript they can 
have that "Oh, that's what they're saying!" experience and go back and hear the 
problematic portion(s) correctly. 

This method was applied extensively in the research. The best use was when it 
was applied as HW or a "warm-up" drill before the actual transcription of a session with 
similar distortions. This method connected the sounds, as they were heard, through the 
distortion to the actual text (Chapter 4). 
Deferred Listening 

This method is both reinforcing and "ear-refreshing," helpful when audio is 
distorted and needs to be heard multiple times to be comprehended. Deferred listening 
offers two options of use: (a) When given a passage already in the instructional program, 
the student listens to the same distorted text at some point after instruction — at home or 
even the next day. Listening without pressure — after the content is known and while the 
ear is fresh — is much easier because the student experiences positive results and 
increased motivation, (b) When applied in the research transcription activity, students 
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used this method combined with the "put-aside-word" method, first listening to the 
passage, then moving to another activity (for example, filling out the feedback form), and 
then listening to the passage again. "Type A" students in particular used this method 
extensively in the research; as they tend to get stuck on a word and need to be reminded 
that until they recognize the word, it helps to move on for the time being and that 
eventually they will get back to it. 
Repeated listening 

The student listens to the entire clip 2-4 times before starting transcription. This 
allows the student to capture the theme, get used to the distortion, and identify repetitive 
words and phrases. This technique was useful for high-difficulty distortions. This 
method was highly effective, and it became the minimum warm-up before the 
transcription, especially when students were introduced to a new distortion type for the 
first time. 

Word segmentation/de-segmentation 

This method was developed by the researcher. It requires understanding of 
language and speaking characteristics, as well as good grammar knowledge, because the 
problem it solves is transcribing two or more words as one word (de-segmentation) or 
breaking one word into two or more words (segmentation). Defining where a word starts 
and ends usually requires knowing the word. However, in the flow of speech there are 
more factors that affect recognition besides having a word as part of a student's active 
vocabulary. For example, when a word is run together with preceding or following 
words, which are its "physical environment," it may connect in the transcriber's ears. 
Some hints that students can use are taken from the language and speaking 
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characteristics, as the student knows which sounds and words have the tendency to be run 
together by native speakers (for example, "I will" is often contracted into "I'll" in 
English). In other cases, grammar knowledge helps as the student can predict certain 
sounds that stem from conjugations and context [for example, if the context is past tense, 
an [ed] can be expected at the end of a verb, even if the verb is not recognized because of 
the physical or "auditory environment" (other sounds/noises)]. Segmentation and de- 
segmentation phenomena were important error types, and students learned how to cope 
with them successfully during the research (Chapter 4). 
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Connecting Student's Challenge with Instructional Solution 

Figure 1 below is based on the writer's experience as well as on interviews with 
other experienced teachers. The approach taken is practical and stems from the student's 
point of view: Namely, it lists the challenge and suggests how to approach it. 



Challenge 


Characteristics of 


Strategy name + How to 




challenge 


address the problem 


-Student hears word(s) 


-Word level 


-Vocabulary Building 


and is able to repeat but 


-All learning phases 


a. Teacher: Prior to listening, 


doesn't understand. 


-Bottom-up 


actively build relevant 




-Cross language 


vocabulary, possibly in context 






or by theme, to alleviate the 






challenge. 






b. Student: Actively build 






vocabulary by writing down the 






word you hear. Then ask the 






teacher or check the dictionary. 


-Student gets "stuck" on 


-Bottom-up challenge in 


- "Put-Aside word" and 


one word and stops 


many cases is caused by 


"Deferred listening" 


transcribing. 


stress and learner's 


techniques 




detail-oriented learning 


- Find the theme technique 




style. 





Figure 1. Teaching and listening methods and how to use them in practice 

(continued) 
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(continued) 
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Figure 1. Teaching and listening methods and how to use them in practice (continued) 
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(continued) 
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Figure 1. Teaching and listening methods and how to use them in practice (continued) 
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(continued) 
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Figure I. Teaching and listening methods and how to use them in practice (continued) 
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(continued) 



Challenge 
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Strategy name + How to 
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Procedures and Data Collection 

Introduction 

The class from which students were invited to participate in this study was made 
up of two sections. Volunteers from each section participated in the experimental 
instruction program in an identical manner. Both sections were taught the same materials 
by the same instructor (the researcher). Total instructional hours for each section: 18. 
Total instructional hours in the entire proposed research study: 36. Total data collection 
period: 1 1 weeks. Additional required hours: one hour for the purpose of inviting the 
students to participate in the study and getting the signed consent forms of those who 
agreed to participate. 

In most cases, weekly instruction occurred twice a week (subject to 
circumstances, such as tests or drills), one instructional hour per day. In all cases, the 
sections were taught on the same day, on consecutive morning teaching hours. 
Distribution of teaching 

The transcription material used in the intervention was taught to all participating 
students by the researcher only, to avoid the possibility of "teacher effects". The 
participants were asked to transcribe the same material that exists in the traditional 
curriculum, with the only difference being that the audio signal was imperfect or 
distorted, as it was modified to represent the less-than-optimum sound quality of material 
that the students are likely to encounter in the future on the job 4 . 

Since this is the first time the researcher was teaching this distorted audio 
material, his presentation was likely to be smoother during the second hour of instruction. 

4 All students, regardless of their rank and future assignment, were trained as voice interceptors. 
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Therefore, to ensure that the quality of instruction was equally distributed over the two 
sections, each time the researcher presented new material he alternated between the 
sections so that each day of instruction one or the other section was the first to be 
instructed. 

Distribution of distortion types 
Main research 

Introduction: The Main Study aimed to answer the main research question of this 
paper, which focused on the effectiveness of listening and instruction methods and their 
contribution to students' ability to cope with three different distortions. A protocol was 
developed to assess (a) how each distortion differed from the other in difficulty and (b) 
which instruction was required to cope with each distortion. At each instruction, all 
students began by transcribing one specific distortion; following that, they were exposed 
to the other distortion types. This practice allowed all the students to get used to all the 
distortion types at the same pace. 

Preparation: All distortion types were distributed over the same text. The 
original (parent) clip was distorted in three ways: Phone Line processing, Static (Pink 
Noise processing), and Babble Noise processing. The study was conducted two days per 
week, one class period per day for each of the two sections involved in the study, over the 
course of three weeks (six sessions). The volunteers in each section were thus exposed to 
the study's modified audio clips in two class sessions per week. Each day (which 
included two instructional hours, one for each section) is defined as a "treatment 
session:" 
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1. In each treatment session, the volunteers in both sections were taught the same 
content (i.e., transcribed audio clips that contain the same text). 

2. The audio clip(s) that the students used for transcription in each treatment session 
were distorted in three different ways (phone-line/pink-noise/babble-noise). The 
types of distortions were distributed equally among the students. In each section, 
only two students heard that day's clip with the same type of distortion. Assuming 
there were six volunteers per section, at the end of each treatment session the 
researcher had four transcriptions transcribed from the "phone-line" version of the 
clip, four from the "pink-noise" version, and four from the "babble-noise" version. 
The distortions were rotated among the students from session to session so that the 
students who transcribed a "phone-line" clip in the first session transcribed a "pink- 
noise" clip in the next, and a "babble-noise" clip in the following one. In this way, 
over the course of three treatment sessions, every participant experienced all three 
types of distortion utilized in the study. 

In the other daily class hours,, participating students were taught the same 
instructional material by teachers who were members of the teaching team. The teaching 
team included the researcher and three other teachers. 
Advanced research 

The advanced study was developed and initiated during the data collection 
process because of the following considerations: 

1 . The data collection for the Main Study was shorter than expected due to a good 
response from the participants, so the researcher determined that there was sufficient 
information to answer the Main Study research question. 
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2. Additional suggestions and questions were raised by the researcher and the 
participants during the main study. An advanced study was a good way to implement 
those ideas and research for the answers. 

3. This part did not exceed the permissions granted for this research; the only changes 
from the original planning were the distortion types and their distribution as described 
below. 

The Advanced Study was developed as a result of the Main Study's initial 
findings and participants' feedback; it focused on exposing the students to more realistic 
and advanced distortions. Thus, its structure was different, as the distribution of 
distortions was different: 

• In each treatment session (each day of data collection) all participants were exposed 
to the same clip (i.e., all students transcribed exactly the same audio distortion and the 
same text). 

• In the following treatment session (the next day of data collection), the same 
distortion was introduced and the text was different. 

• The same pattern was repeated over three to four sessions, until the transcription 
results showed that the students had attained good proficiency in transcribing this 
specific distortion. 

• Another distortion was introduced, with new text for each treatment session, as 
described above. 

Overall, three distortions were introduced in this phase of the research, over the 
course of 8 weeks and 12 sessions: "Competing Speech" process, "Muffled" process, and 
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"Chopped" process. See "Definitions" section in Chapter 1 and "Technical Procedures of 
Audio Clip Preparation" in Appendix 3 for more details. 

As in the Main Study, data collection sessions were conducted two days per week, 
one class period per day, for each of the two sections involved in the study. The 
volunteers in each section were thus exposed to the study's modified audio clips until 
they had mastered proficiency in the presented distortion, and then that distortion was 
replaced with another distortion. 

Weekly Instructional Plan and Data Collection 
The researcher conducted the experimental treatment twice a week, one 
instructional hour (50 minutes) each time, for 11 weeks. As expected, instruction was 
tailored and modified throughout the different research phases as a result of (a) lessons 
learned, (b) the progression of the research, and (c) the changing structure and 
distribution of distortions. 
Main Research 

In each instructional hour 
1 . The session started with the teacher (researcher) instruction that addressed the 
challenges he identified in the student transcriptions from previous sessions (except 
for the first session). For example, when the researcher identified segmentation 
problems, he addressed the challenge and taught the students how to cope with the 
challenge, using a variety of instructional techniques according to the nature of the 
challenge and students' needs (see "Instructional methods" section of this chapter). 
The researcher employed instructional methods according to individuals' needs, as 
different students had different challenges, which resulted in the employment of 



different instructional techniques. Methods were selected and tailored according to 
the progress of individual students throughout the course. 

Students were tasked to warm-up for the distortion they were going to transcribe in 
the current session (except for the first session). Students used clips that were already 
available from previous sessions. For example, if student A was tasked to transcribe 
a phone line distortion on a certain day, s/he was tasked to warm up on a phone line 
distortion from previous sessions that included text s/he had already been exposed to 
in the past. 

Students transcribed their audio clips with their assigned distortions of the day and 
text they had not yet encountered. Each student was given control of his/her audio 
player. Students were able to rewind as necessary, for an equal amount of defined 
time, approximately 20 minutes, according to the progression of the course and 
students' skill levels. As in regular test administration practice, students were not 
supported by active teaching or help, unless there were technical issues. 
At the end of each transcription session, the data collection form was filled out and 
submitted to the teacher (see Appendix 2 for an example of a weekly data collection 
sheet). Students provided feedback regarding 

a. Their perception of the task difficulty, their own stress level during the task 
performance, and their confidence of the accuracy of their transcription (5- 
point Likert-type scale). 

b. Students reflected on the task and the effectiveness of the instructional 
methods (open-ended question). 



c. Students reported which listening/transcription/learning methods they used on 
their current transcription. 

d. To give the students more experience, improve their listening flexibility, and 
expose them to more distortions in a shorter period of time, they were tasked 
to listen to the other clips of the day (same text of the day, distorted with the 
other distortion types or the original clip). The students were tasked to 
estimate which of all the distortions heard in this session was most difficult. 

e. Based on the printed full original text ("solution"), provided by the teacher, 
and the optional use of all clips including the non-distorted parent clip, 
students were tasked to translate the transcribed text into English. This step 
was a control check for the teacher in his role of researcher to verify that the 
text was not beyond students' lingual level. In addition, students were asked 
to write down words they (a) did not know during the transcription or (b) still 
could not identify or understand after listening to all other clip versions. 
These words were used to estimate the "Unfamiliar Word" error type. 

After feedback submission, and when time allowed, students asked questions 
regarding the day's activity and the instructor helped them as needed. 
Following some of the class sessions, the researcher assigned targeted follow-up HW 
according to his discretion and his findings from the feedback and transcriptions. 
Homework was collected and evaluated by the instructor but was not part of student 
evaluations in the course, because the research was not part of any grading of 
students. This HW assignment practice was found to be effective. Student 
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transcription results showed interesting findings that were attributed to the assigned 

HW (see Chapter 4: "Results and data analysis"). 

At the last session of the Main Research, students filled out a more 
comprehensive, open-ended feedback, where they were asked to (a) reflect on their 
experience, (b) estimate their own progress and advancement in transcription, (c) either 
recommend or not recommend such activity for other classes in the future, and (d) 
communicate whether or not they wished to continue with the more advanced research 
(see below). All students requested to continue. 
Advanced Research 

In each instructional hour 

1 . The session began with teacher (researcher) instruction to address the challenges he 
identified in student transcriptions from previous sessions (see "Main Research" 
above). This part was optional, according to the teacher's discretion and findings 
from the previous sessions. 

2. Students started the transcription session with (an optional) warm-up — when the 
distortion of the day was not new to the students, they were tasked to warm up on 
previous clips of the same distortion. Students were provided full texts (solutions) for 
all previous clips and the general directive was to conduct a "listen while reading" 
(text-accompanied audio) drill. In the case of challenging distortions, students were 
provided with a "transcription puzzle" drill option, which was found to be extremely 
helpful, particularly for those students who were more challenged. 
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3. At the end of each transcription session, data collection forms were filled out and 
submitted to the teacher (see Appendix 2 for an example of a weekly data collection 
sheet of the advanced research phase). Students provided feedback regarding 

a. Their perception of the task difficulty, their own stress level during the task 
performance, and their confidence of the accuracy of their transcription (5- 
point Likert-type scale). 

b. Students reflected on the task and the effectiveness of the instructional 
methods (open-ended question). 

c. Based on the printed full original text ("solution"), provided by the teacher, 
and the optional use of the non-distorted parent clip, students were tasked to 
translate the transcribed text to English. This step was a control check for the 
teacher in his role of researcher to verify that the text was not beyond 
students' lingual level. In addition, students were asked to write down words 
they were not yet able to identify or understand. 

4. After feedback submission and when time allowed, students asked questions 
regarding the day's activity, and the instructor helped them as needed. 

5. Following some of the class session, at the researcher's discretion and according to 
his findings from the feedback and transcriptions, students were assigned follow-up 
HW geared to solve challenges in their transcriptions. Homework was collected and 
evaluated by the instructor, but it was not part of the student's evaluation. The most 
common homework was a transcription puzzle of the challenging distortion. 

At the last session of the Advanced Research, students filled out a more 
comprehensive, open-ended feedback page, where they were asked to reflect on their 
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experience, to estimate their own progress and advancement in transcription, to 
recommend or not recommend such activity in other classes in the future, and to express 
whether or not they wished to continue with this Advanced Research. Students could 
recommend stopping the study; and they were given a choice of different possible 
research plans. 

Data Collection and Analysis 

Introduction 

Data was collected over the course of 1 1 weeks, throughout the research. During 
weekly instruction, data collection sheets were collected and processed before the 
following instruction, in order to apply lessons learned and conduct corrective 
instruction. These sheets contained transcriptions and student perceptions, as described 
above and in Appendix 2. 

Weekly Data Collection and Analysis — Main Study 

Coherency/accuracy — numerical grading of transcriptions. Transcriptions were 
scored for accuracy/coherency; scoring emphases were (a) the precision of listening, (b) 
the extent to which the message was conveyed, and (c) the error types. Scoring was done 
according to the following guidelines: 

1. In this "transcription-driven evaluation" approach, the emphasis in evaluation was on 
listening accuracy and error types. For example, a spelling error (in English: "KA" 
instead of "CA," or "OL" instead of "ALL") was not counted and did not reduce the 
transcription's accuracy score, as it is not considered a listening error. However, 
connecting words (writing two words or more as one) or breaking one word into two 
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is a listening error, although less severe than the inability to hear certain sounds such 
as the Israeli "R" or "CH" (pronounced like the "X" in the Spanish word "Mexico"). 
2. Scoring was performed as follows: 

a. For the purpose of error count (accuracy), a word that contained an error (as 
defined in the "transcription-driven" approach above) was counted as one 
error without regard to how "serious" an error was (i.e., how "far" it was from 
the correct word). In this way, the researcher was able to calculate the 
percentage of words transcribed correctly or incorrectly. 

b. For the purpose of numerical scoring of the whole transcription quality 
(coherency), error severity was taken into account; each error was scored 
according to how far it was from the correct word and to how the main 
message was conveyed. The text was broken into sentences, and each 
sentence was evaluated for its accuracy [%]. The final accuracy value was the 
average of all sentence accuracy values. 

The error count measurement did not show to be representative of quality; 
therefore, it was dropped in the Advanced Study. However, the coherency 
measurement was found to be representative of the overall transcription quality; it 
was adopted in the Advanced Study as the main tool for assessment and 
evaluation of transcription quality. 
Error types — qualitative evaluation of transcriptions. Transcriptions obtained from 
the participants were analyzed comparatively to find common trends in transcription 
errors within the different distortion types. Different error types suggested which 
teaching practices worked better for coping with the different distortions. The error types 
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also suggested students' challenge areas and possible improvements in instructional 
methods. During the research, different methods were employed and developed 
(Chapter 4). 

Student perception feedback analysis. Feedback was analyzed qualitatively to find 
common trends and patterns in student perceptions and opinions. The researcher used 
data and analysis results to assess student perceptions regarding (a) their confidence in 
the accuracy of their transcriptions, (b) the difficulty of the distorted audio materials, and 
(c) the stressfulness of the transcription tasks. The researcher also sought to identify 
other possible needs that might require future attention, as well as any new ideas or 
insights of students that they believe might help them cope with listening challenges. 
Information from student feedback was used to improve the next week's instruction 
(Chapter 4). 

Instructional methods efficiency feedback analysis. Feedback was analyzed 
qualitatively to find common trends and patterns in student use regarding the various 
instructional methods available to them. Data and analysis results were used by the 
researcher to obtain information regarding student preferences and extensiveness of use 
throughout the research (Chapter IV). 
Weekly Data Collection and Analysis — Advanced Study 

The advanced study grading focused on the measurements that proved to be most 
representative of transcription quality in the main study. In light of the students' 
feedback and the researcher's analysis, the data collection sheets were shortened to 
include: 

1. Transcriptions that were graded for coherency only. 
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2. Student perceptions (indicators) of difficulty, stress, and confidence of transcription 
quality (5-point Likert-type scale). 

3. Optional open feedback. 

4. Translation of script, for level of text control. 

Coherency. The text was broken into sentences, and each sentence was evaluated 
for accuracy [%]. The final accuracy value was the average of all sentence accuracy 
values. 

Student perception (indicators). Feedback was analyzed qualitatively to find 
common trends and patterns in student perceptions and opinions. The results showed 
interesting patterns throughout the research (Chapter 4). 

Protection of Human Subjects 

The study was non-intrusive and non-destructive regarding the normal day-to-day 
classes and instructional activities of the Basic Program curriculum. This study was 
approved by both the Institutional Board Review of the U.S. Army Defense Language 
Institute Foreign Language Center (DLIFLC) and the Argosy University San Francisco 
Bay Area Institutional Board Review. 

Risk-Benefit Considerations 

Conducting the study required approximately one instructional hour for each data- 
collection session and application of teaching strategies. In terms of scheduling, the team 
leader needed to assign just two hours for each above-described weekly activity. 

All the above hours had value for teaching and learning. The study had no effect 
on other DLIFLC operations. No other DLIFLC resources were needed to conduct the 
study. 
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The benefit of the study is a potential improvement in both teaching and 
learning — as well as possible implementation of the findings — in future classes in 
Hebrew and other languages. 

Analysis 

The research question and additional substantive questions were answered by (a) 
feedback provided by students regarding their perception, (b) a qualitative analysis of the 
typical errors in transcripts from both the study group and the control group, and (c) 
grading of the transcripts. 

RQ: Which teaching materials and teaching techniques appear to be most 
effective in helping learners acquire the ability to cope with low-quality audio? 

• Data was extracted from feedback provided on the weekly and final data collection 
sheets. Students' thoughts and preferences were qualitatively analyzed for common 
trends of preferences and experiences. 

• Microsoft Excel® software was used for analysis. 



82 

CHAPTER FOUR: RESULTS AND DATA ANALYSIS 
Purpose and Design of the Study 

The purpose of the study was (a) to develop imperfect audio materials 
appropriate for early-stage learners of Hebrew and (b) to evaluate the effects of the 
instructional plan on the ability of such students to cope with the materials. The study 
involved exposing learners at an early stage of instruction to distorted audio material, 
while applying specific teaching techniques to help them cope with the challenge. 

Regular analysis of the feedback and results in real-time (after each lesson) 
enabled the researcher to modify each lesson according to the feedback and results of the 
previous lesson. The result was that it improved both student performance and the 
efficiency of the instructional plan (see Chapter 3). 

Chapter Overview 

This chapter presents results of the research in a descriptive format, including 
tables, figures, narratives, trend markers, and descriptions of emerging trends. It 
concludes with a summary of findings and results. Results and analyses are divided into 
sections according to how they were obtained and their characteristics: 
1 . Results derived directly from student feedback: 

a. Open-ended feedback, where students wrote their opinion 

b. 5-point Likert-type scale used to assess student indicators pertaining to 
perceptions of stress, difficulty, and confidence levels 

c. 3-point Likert-type scale (Yes/Some/No) where students indicated if they used 
teaching methods on the current transcription, and if so, which method was used 
and to what extent 



83 

2. Results derived from researcher's evaluation of student transcriptions: 

a. Error types 

b. Numerical transcription evaluation by the researcher — measuring 
accuracy/coherency of transcripts (see definition) 

Description of Respondents 

The population of this study consisted of all DLI Hebrew Basic Program students 
(N = 30-40), enrolled in classes 21501HE00212/3 12/412/1 13/213. The sample in the 
preliminary study (n = 6) consisted of students from section A of class 21501HE00412, 
who began the Basic Program in June 2012. The sample for the Main and Advanced 
studies (n = 12) consisted of students from class 21501HE00113, who began in October 
2012. These students are the people who volunteered to participate in the study, 
comprising the class that yielded the results presented in this paper. 

The program is located at the Presidio of Monterey, California. Students in the 
program are adult learners, ranging in age from 18 to 35, who are United States military 
service members or government agency employees. 

The selection of these specific students was a convenience sampling, because the 
researcher was part of the teaching team for the class. The students were assigned to the 
class by the DLI, using the DLI's standard practice and language assignments in 
accordance with standard procedures, and were assumed by the researcher to be 
representative of typical students in the Hebrew Basic Program. Based on student 
achievements in the course, such as GPA and other standard measures, the researcher 
confirms that his assumption is valid: Students were found to be representative of a 
typical class. 
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Instrumentation Description 

Audio material was selected from the standard teaching material available and in 
use in the Basic Program curriculum. In the Main Study, selected audio clips, distorted 
by the addition of Pink Noise, Babble Noise, and Phone-Line Processing, were used for 
transcriptions; in the Advanced Study, further additions included Competing Speech, 
Muffled distortion, and Chopped distortion. For more detail, see the "Definitions" 
section in Chapter 1 and "Technical Procedures of Audio Clip Preparation" in 
Appendix 3. 

The researcher was the sole instructor. He used a variety of instructional 
methods, detailed in Chapter 3, to support student learning and the writing of 
transcriptions. All methods and strategies were available to all participants and were 
employed by the researcher according to student needs and the challenges presented to 
them by the different characteristics of the audio materials they processed. During the 
research the instructional experience yielded the specialized teaching techniques reported 
in this paper. Other methods, intended to be used but found to be less effective, are 
reported in this paper as well. 

Data collection was done using Data Collection Sheets (DCSs); it was recorded 
and analyzed simultaneously, using Microsoft Excel, in order to improve the next 
instruction. This dynamic nature of study and instructional plan drove continuous 
changes in the DCSs, which were continuously adapted throughout the research to match 
the ongoing developments in the study. For more details see Appendix 2: "Data 
Collection Sheets." 

Microsoft Excel was used for all descriptive and inferential data analyses. 
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Variables 

Dependent Variables 

Dependent measured variables were designed to answer the research question, 
which addressed instructional effectiveness. The variables were derived from both 
student perception and the analysis of transcription results: 

1. Student perception indicators were used to assess: 

a. The instructional method that seemed to work best 

b. Level of stress the student felt during transcription 

c. Student confidence in the accuracy of transcription 

d. Perception of student's difficulty in transcribing the distorted audio 

2. Analysis-based variables: 

a. Number and nature of transcription errors 

b. Coherency/accuracy: the native reader's ability to understand the transcriptions 

c. Number of instructional methods used 

d. Types of instructional methods/techniques used 

e. Extensiveness of instructional methods/techniques used 
Independent Variables 

Independent variables were the different distortion types the students were 
exposed to. The distortions for the main research were selected in advance; as the 
research progressed, more distortions were developed and introduced. The pre-selected 
distortions for the Main Study were Pink Noise, Babble Noise, and Phone-Line 
Processing. The developed distortions for the Advanced Study were Competing Speech, 
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Muffled, and Chopped. For more detail, see the "Definitions" section in Chapter 1 and 
"Technical Procedures of Audio Clip Preparation" in Appendix 3. 

Research Question 
The research question in this study was: 

RQ: Which teaching materials and teaching techniques appear to be most 
effective in helping learners acquire the ability to cope with low-quality audio? 
Effectiveness of Teaching Techniques 

Information regarding method use was obtained from students using (a) Data 
Collection Sheets (DCSs), where students reported on a 3-point Likert-type scale 
(Yes/Some/No) whether or not they used each teaching method on the current 
transcription and the amount of use, and (b) open narrative feedback. 

The full 3-type scale DCS is described in full in Appendix 2. For the students' 
convenience and recognition of techniques, each teaching method is described by a 
narrated sentence that represents or demonstrates the method, following the underlined 
name of the method. Throughout instruction, the researcher used the method names and 
examples to connect the actual method name and its practice. Figure 2 demonstrates a 
portion of a Data Collection Sheet: 



Metacognition It helped me understand the process of what is 
happening during transcription. 


No / Some / Yes 


Vocabulary I feel better when I know the words I transcribe. 


No / Some / Yes 



Figure 2. Demonstrational partial Data Collection Sheet for listening methods. 
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Reported perception of students 

Overall use of techniques 

The 3-Point Likert Scale index was used by the researcher to estimate student 
response to the instructor's teaching; the researcher's focus was varied methods with 
different styles and approaches. 
Presentation of data 

For each session, the average number of listening/transcription methods used per 
student was calculated. Table 1 presents the average number of "Yes," representing 
substantial and self-aware use of methods by students, and "Some," representing less- 
extensive and only possible use. 

Table 1 

Use of Listening Techniques 



Session "Yes" "Some" Total Use 

1 8.0 4.6 12.6 

2 6.2 5.7 11.9 

3 5.5 4.4 9.9 

4 6.3 6.8 13.1 

5 7.1 5.9 13.0 

6 7.8 4.5 12.3 



Note. Average per session per student 

Initial reported use of techniques was high. A decline was recorded in the second 
and third sessions, yet the total number increased, stabilizing at approximately 13 
techniques, towards the end of the study (Figure 3). Considering the extensiveness of use 
(reports of "Yes" versus "Some"), the general pattern in the first three sessions can be 
described as fluctuating. This can be ascribed to students' lack of experience with the 
different methods used in these sessions and their tendency to try many of the techniques. 
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Beginning with Session 4, students focused on more effective methods. In addition, their 
reports may be considered more reliable; because, by that time, the researcher had more 
experience, and the difficulty of audio materials was adjusted. Another factor that should 
be considered is that after three sessions the students completed a full cycle of distortions: 
Each student had already experienced all three distortions equally. 



Average of methods used per student 
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Figure 3. Total use of techniques by students. 



For these reasons, the analysis that began in Session 4 is considered by the 
researcher to be more meaningful. This analysis shows an increase in use of methods 
reported as "Yes," indicating that the students identified them as more effective, and a 
decrease of methods reported as "Some." This pattern can be attributed to two main 
factors: (a) students became more aware of, more familiar with, and more experienced 
with the different techniques; and (b) during the research additional methods were 
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introduced, resulting from student suggestions and the researcher's attempt to alleviate 
student challenges. Figure 4 demonstrates that student focus on the more effective 
methods increased during the last three sessions. 
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Figure 4. Total use of techniques by students, Sessions 4 to 6. 



Figure 4 shows an increasing number of students who reported more extensive use of 
transcription and listening methods ("Yes"). Reports of "Some," which indicated less 
extensive use, was reported more in the earlier sessions and dropped off as the research 
advanced. The researcher attributes this trend to students who gained experience of 
"what works and what doesn't work" by trial and error. As the study progressed, 
students knew which methods worked for them: They dropped the methods that did not 
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work for them. It is probable that some "Some" responses became "Yes" responses as 
students determined the effectiveness of the various methods. 



Most used techniques 
Main study 

This index was used by the researcher to identify student preferences for transcription 
methods. Students' narrative responses indicated their ability to effectively combine 
techniques to improve their transcription results. 
Presentation of data 

At each session, students reported what were the most used methods, indicated by 
"Yes" on the DCS. Table 2 and Table 3 show the number of times each transcription 
technique was utilized by the students. 



Table 2 

Reported use of Listening Techniques 1-9 



Session 


Metacognition 


Vocabulary 


Repeated 
Listening 


Bridge the 

gap, 
synonym 


Put-aside 
words, get 
back later 


Theme & 
key-words 


Grammar: 
note the 
noun 


Grammar: 
note the 
verb 


Grammar: 
note the 
adjective 


1 


33% 


83% 


% 

0% 


42% 


83% 


92% 


33% 


42% 


33% 


2 


45% 


73% 


0% 


9% 


45% 


82% 


45% 


45% 


45% 


3 


25% 


100% 


0%* 


0% 


25% 


67% 


17% 


50% 


33% 


4 


25% 


83% 


58% 


17% 


50% 


75% 


42% 


42% 


42% 


5 


33% 


100% 


67% 


0% 


33% 


67% 


42% 


58% 


33% 


6 


50% 


100% 


58% 


8% 


50% 


75% 


42% 


83% 


50% 



Note. Average per session. Method not introduced 
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Table 3 

Reported use of Listening Techniques 10-18 



Session 


Language 
characteristic: 
writing 


Language 
characteristic: 
speaking 


Language 
characteristic: 
pronunciation 


Write the Defer 
best you listening 
can 


Listening List, while 
while reading & 
reading speaking 


Punctuation Transcription 
Puzzle 


1 


67% 


42% 


42% 


75% 


17% 


50% 


17% 


17% 


33% 


2 


27% 


27% 


45% 


64% 


9% 


18% 


9% 


9% 


18% 


3 


17% 


33% 


50% 


50% 


25% 


17% 


8% 


8% 


25% 


4 


25% 


33% 


25% 


50% 


25% 


17% 


8% 


0% 


8% 


5 


58% 


33% 


42% 


25% 


8% 


42% 


25% 


8% 


33% 


6 


42% 


42% 


50% 


50% 


0% 


25% 


17% 


25% 


8% 



Note. Average per session 



Figure 5 presents the breakdown of overall use. It shows the students' tendency 
over all six sessions to use different listening and transcription techniques: 



Average percentage of techniques use by 
students per session (all sessions) 




Figure 5. Breakdown: students' extensiveness use of techniques, overall study. 
Note that the Repeated Listening method was not introduced until session 4; the 
presented percentage represents Sessions 4-6 only. 

Figure 5 shows prominent techniques (cut-off line was set at 60%) as 
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(a) familiarity with vocabulary (90%), (b) understanding the theme (76%), and (c) 
repeated listening (61%). The Repeated Listening technique was introduced to students 
in the fourth session, and it became one of the overall leading techniques. 

These methods represent different strategies: Vocabulary knowledge is a bottom- 
up method, while theme understanding is a top-down method. This combination is a 
good balance of strategy use by students; they employ both kinds of strategies in order to 
support learning. Because Figure 5 represents the average use over the course of the 
entire main study, it includes the span of time students needed to get used to the methods 
as well as the influence of the learning habits they brought into the research. A 
comparison between the first three sessions and the last three sessions is in place. In 
addition, it can be interesting to see how the introduction of the Repeated Listening 
method, a top-down strategy, influences the utilization of theme knowledge. Table 4 and 
Table 5 present the average use of methods in Sessions 1-3 and 4-6: 



Table 4 

Use of Listening Techniques in Sessions 1-3 vs. 4-6 



Session Metacognition Vocabulary 


Repeated 


Bridge the 


Put-aside 


Theme, key- 


Grammar: 


Grammar: Grammar: 




Listening 


gap, 


words, get 


words 


note the 


note the verb note the 






synonym 


back later 




noun 


adjective 


1-3 35% 85% 


0% 


17% 


51% 


80% 


32% 


46% 37% 


4-6 36% 94% 


61% 


8% 


44% 


72% 


42% 


61% 42% 



Note. Average per session. Method not introduced 



Table 5 

Use of Listening Techniques in Sessions 1-3 vs. 4-6 



Session Language 


Language 


Language Write 


Deferred 


Listening 


Listening 


Punctuation Transcription 


characteristics: 


characteristic: 


characteristic: the best 


listening 


while 


while 




Puzzle 


writing 


speaking 


pronunciation you can 




reading 


reading & 
speaking 






1-3 37% 


34% 


46% 63% 


17% 


28% 


11% 


11% 


26% 


4-6 42% 


36% 


39% 42% 


11% 


28% 


17% 


11% 


17% 



Note. Average per session. 
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Figure 6 presents the changes in method use. Use of theme recognition dropped 
by 8% and gap bridging by 9%; the two strategies together were a combination of 
bottom-up and top-down strategies on the sentence level. The introduction of "Repeated 
listening," a new top-down technique in which students were asked to listen to the whole 
clip before starting to transcribe, may have contributed to the changes in method use. 
Some students had already identified the theme before starting to transcribe, so their use 
of other similar strategic methods became obsolete. A good representation of this trend 
is a student who wrote in Session 4, "The new phone line can distort some words heavily; 
because we listened to it a few times before transcribing, it was easier to transcribe [new 
technique of repeated listening]." 

Students simultaneously increased their use of bottom-up strategies. Vocabulary 
knowledge increased by an average of 9% (Figure 6) and, in fact, vocabulary use 
increased by 100%, as reported in the last two sessions (Table 2, Table 1). This trend can 
be attributed to the nature of distorted audio, as students tend to focus on one specific 
single word at a time through the distortions. 
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The change in use of methods, sessions 1-3 vs. 4-6 




Figure 6. Students' Extensiveness of techniques use, change from sessions 1-3 to 4-6. 

A noticeable increase in using grammar-based and language-based techniques 
reflects the emerging trend of increased use and focus on single words. Students develop 
their skills in such techniques over time, as they progress in grammar, language construct, 
and syntax. In addition to this natural trend, the more the students are challenged by 
distortions and have difficulty understanding the words, the more they have to rely on the 
words they do hear and are able to understand. They needed to focus on the sub-word 
level and its grammatical nature to draw conclusions. In Session 4, a student described 
his progress: 

This was much more difficult to hear, so I don't know necessarily how many 
words were new. However, listening carefully, I was able to glean some more 
information than I did initially. Also, I was able to understand SOME of it based 
on grammar. I didn't hear EVERYTHING, but the words fit better due to my 
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knowledge of grammar. Also, I listened specifically for words that would FIT 
grammatically. I think I did more poorly on this one, but liked it a lot! 

Grammar use became of major importance in the last sessions; the more the students' 

proficiency in the methods increased, the more they used the methods. Use of the "Note 

the verb" method increased to 61% (Figure 7, Table 5) and to 83% by the end of the 6 th 

session (Table 2). Five out of six grammar- and language-driven methods showed 

increases of 2-15% (Figure 6). 



Average percentage of techniques use by 
students per session (sessions 4, 5, 6) 




Figure 7. Breakdown: students' extensiveness use of techniques, Sessions 4-6. 



Metacognitive process is yet another factor that is strongly demonstrated in the 
presented findings; in their narratives, students displayed high consciousness regarding 
their learning process and were able to analyze it clearly using professionally acquired 
terms. The reported use of metacognitive process was stable although not very high 
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(35%, see Figure 5); yet the abundance of indirect data in the narrative points to the 
possibility that the use of metacognition was underreported by the participants. 

The following quotes from student feedback in Sessions 5 and 6 demonstrate how 
adult learners who control and regulate their learning process use a combination of Top- 
Down and Bottom-Up strategies, swing back and forth between the two concepts, and 
move among all capacities in order to create the best transcription possible. It is 
important to note the level of self-consciousness and the role that metacognition plays in 
the descriptions. The use of professional and conceptual terms in student descriptions is 
indicative that, during this research, students (a) expanded their learning process and (b) 
developed their ability to implement the strategy tools they acquired: 

Normally, I just listen, and don't try to think. I just let my subconscious absorb 
what I hear. Then I listen for any words that I couldn't hear or didn't know. I 
listen more carefully to determine if it's just a matter of difficulty hearing a 
known word or if it was an unknown word. Then, I use the theme that I perceive 
to check what I heard, and make sure it makes sense, and check for agreement in 
the grammar. Then, I go ahead and check the meaning for coherency. If 
something seems wrong, I go back and listen again and pinpoint the mistakes. 
(Student 1). 

Another student describes how she incorporates techniques of different strategies 
together: 

My listening technique: I generally will listen to the whole thing first so I know 
where words start and stop. I also listen for grammar tense and gender. I also 
listen to the theme and vocabulary, I map out the story so I know what I'm writing 
[Top-down]. The second run through, I play and pause the audio in word clusters 
of 3-5 words and transcribe what I hear exactly how I hear it. In the last run, I 
listen while reading what I'm writing and make corrections as I see fit using all 
the information I collected previously. (Student 2). 



97 



A third student describes the techniques he puts to work, starting top-down and gradually 
incorporating a bottom-up approach: 

Used fill the gaps. My technique: In my listening, I focus on logic, I'm not 
focusing on verb conjugations or masculine feminine, I'll work on that after I 
know the word. I focus on "does it make sense." I listen to the passage 3-5 times 
and try to make sense of it. On the 4th-5th time, stopping when I hear an 
unfamiliar word and trying to see what fits, what would I say there? And then I 
transcribe as I'm doing this because I'm a visual learner and I picture it. I try to 
transcribe the unknown words because I might know it by sight and phonetically 
spell. Then after I have transcribed I listen and make sure what I wrote 
corresponds to what is being said. 

These narratives vividly demonstrate how a balanced, self-aware learner utilizes a diverse 
supply of available tools and tailors his/her work plan according to self-recognized needs. 
The learner explains how s/he moves from the lower-than-sub-word level (agreements of 
words and conjugated words, according to grammatical rules) to the beyond-paragraph 
level (coherency and main theme) and then tails back to a sentence level (grammar and 
coherency). While being aware of one's own learning style, such as being a visual 
learner, and while expecting the text to repeat later in the audio, students use acquired 
spelling, auditory, and writing methods related to the learned foreign language and 
alphabet, to write down unknown content, such as an unfamiliar word. Writing down a 
word without understanding it is against the nature of an untrained person; it requires the 
conscious ignoring of cognitive needs by writing down what seems senseless at 
the moment. 

Some students display a less rounded approach and use fewer, less diverse 
methods. Their number was small, but they needed a more focused effort by the teacher 
to utilize more techniques. The following quotes represent this smaller group of students: 
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My listening technique: I just listen and write what I hear. If I understand the 
context of the story I can narrow down what words might be if I can't understand 
something. Other than using context I just listen and transcribe. 

Another student wrote, "My listening technique: Just listen — pay attention to theme, 
tense, person, and know the words." These students use mostly bottom-up techniques, 
with a focus on known words. In most cases these students showed lower performance 
on their transcriptions. 

Advanced study 

Although the use of techniques was not collected in the Advanced Study, 
students' verbal and written feedback indicates that use of the following transcription 
techniques increased: repeated listening, deferred listening, write the best you can. The 
researcher, while teaching before and after each transcription session, extensively used 
the following methods for homework as well as warm-up in the classroom: transcription 
puzzle and listening while reading. Student feedback confirms the effectiveness of 
these techniques. 
Student indicators 

Student indicators refer to three indices the students were asked to report on a 5- 
point Likert-type scale, used during each transcription session to assess (a) their 
perception of stress during each transcription session, (b) their difficulty to transcribe, 
and (c) their confidence levels of transcription accuracy. 
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Presentation of Data 

Table 6 

Student Perception Indicators 



Session Difficulty Stress Confidence 

1 16 ZO 32 

2 3.4 2.9 1.9 

3 3.9 2.8 2.0 

4 3.8 3.1 2.6 

5 2.9 2.4 3.1 

6 2.7 2.3 3.2 



Note. Average per student. Range of scale: 1-5 

Table 6 presents the average report of each measure at each session. It 
demonstrates how, at the beginning of the research, the difficulty level for this class had 
to be set by the Signal-to-Noise Ratio (SNR) and a change in the phone distortion. 
Figure 8 shows how student indicators fluctuated in the beginning as corrections in SNR 
were made. The SNR level was correctly set for the third session, and, at that time, 
students reported the greatest difficulty, highest stress, and lowest confidence in their 
performance, indicating that when students are challenged by the distortions they are in 
the best position to improve their performance. 
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Students' Indicators, Overall Sessions 




0.5 

0.0 — 

Session 1 Session 2 Session 3 Session 4 Session 5 Session 6 
— ♦ — Difficulty ^< -Stress A Confidence 

Figure 8. Students' average perception throughout the main study. 

Starting in Session 4, data can be considered to be more reliable. At this point of 
the research, all participants had been through a full cycle of distortions and had all been 
equally exposed to all three distortions. Adequate difficulty levels of SNR and phone- 
line distortion had been established; students were already familiar with listening and 
transcription techniques and had applied them to their work. Student narratives reflect 
the same picture, and most of them reported in the fourth session that this difficulty 
setting was indeed challenging: "I could barely hear any of the words. Most of what I 
heard was cognates of English. Everything else is a guess. The babble sound "ate up" 
most words," and "Static was pretty difficult. Some letters/sounds got lost and it stressed 
me out hearing only fragments of words. . ." There is a relationship between the difficulty 
of the audio and student stress, a trend reflected in Figure 9. 
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Student Indicators, Sessions 4-6 
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Figure 9. Students' average perception, Sessions 4-6. 

Figure 9 focuses on the collected data from Sessions 4-6. In Session 4, students 
were introduced to audio of an increased difficulty level; they reported higher stress and 
greater difficulty in transcribing, while at the same time they reported lower confidence. 
In Sessions 5 and 6, their reports of difficulty decreased and their reports of confidence in 
production increased. The same pattern emerges from the students' open-ended 
verbal/written feedback in Session 6, as the student connects between his confidence 
level and his stress and reports that he is not challenged any more: 



When I'm confident I stress out less, to the max! In this passage I understood 
almost every word, although "a ,l 7l5;" cannot be the word. It may be a ,l 7ix but I 
don't know what it means. Some kind of task? A job? Something that needs to 
be finished in less and less time.. 
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Another student reflects: "Static — this one was easy, yet constructive. You're 
pinpointing/polishing the skills of the students in their listening and how to accurately 
annotate what they hear. I tend to hear things rather easily (comparatively). . ." while 
another suggested increasing the distortion even more: 

This was a fairly easy transcription exercise. The distortion did not hinder my 
ability to hear the passage. The exercise would have been more a useful 
experience if the content were a little more challenging or the distortion was 
greater. Preferably greater distortion. 

A fourth student writes: "It's getting easier at this point. I think I've found a technique to 
listening." The decrease in students' challenge in the fifth and sixth sessions, shortly 
after they had reported high challenge in the fourth session, which was at a similar 
difficulty level, as well as an increase in their measured quality performance as reported 
below indicated a very rapid learning curve and an efficient use of transcription and 
listening techniques. This prompted the decision to end the main study and develop the 
advanced study, as reported below. 
Numerically measured performance of students 
Errors 

In addition to student perceptions of the effectiveness of teaching, indications for 
changes in student proficiency were collected using analysis and quality evaluation of 
their transcriptions. Three main indicators represent transcription quality in this paper: 
(a) simple error count, (b) four prominent error types in transcription, and (c) the ability 
of a native speaker to understand the produced text, which is referred to in this paper as 
"coherency." In contrast to student perceptions, this information was not solicited from 
the participants but was generated by evaluating and numerically grading the texts. 
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Error counts 

A simple count of errors was recorded for each transcription. Errors were marked 
according to the protocol described in Chapter 3. Table 7 presents the average number of 
errors per transcription throughout the Main Study. 

Table 7 

Transcription Average Errors, Sessions 1-6 



Session Average number of errors per transcription 

1 1033 

2 10.54 

3 10.91 

4 10.25 

5 5.91 

6 6.58 



Note. Average per student. 

Figure 10 shows a high count of errors throughout Sessions 1-4 and a drop of 40- 
50% in Sessions 5-6. The counts of the first two sessions remained high, even though the 
audio clips used in these sessions were less distorted. These findings are easy to connect 
with the exposure of students to the distortions and instruction they received in Sessions 
1-2: Despite the increase in the difficulty of distortions, the number of errors remained 
stable rather than increasing. The students adjusted to the distortions in Sessions 3-4, and 
a decline in error count was observed in the last two sessions. 
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Figure 10. Average number of errors in each transcription at each session. 



Prominent Error types — overall count 

Four main types of errors were observed and recorded by the researcher as the 
transcripts were evaluated. The error types were (a) totally distorted (TD) words, (b) 
transcription errors stemming from words unfamiliar to the students (UNF), (c) 
transcription errors stemming from segmentation (Seg), and (d) transcription errors 
stemming from bad spelling (IS). When an error type was counted three times or more in 
a single transcription, it was counted as a "prominent error type." Each transcript could 
be characterized as showing as many as four prominent error types. During instruction, 
students were corrected, made aware of the typical errors and the reasons leading to them. 
In addition, students were assigned corrective activities to support their learning. 
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Presentation of data 

Table 8 presents the number and percentage rates of prominent error types at each 
session throughout Sessions 1-6 and their breakdown into the different categories. For 
example, in Session 1,11 out of 12 transcripts (92%) showed segmentation problems and 
all transcripts showed errors emerging from unfamiliar words (100%). In all the 
transcripts together, 32 appearances of prominent errors of all types were counted in 
Session 1 (out of 48 possible in 12 transcripts), with an average of 2.67 error types per 
transcript. 

Table 8 

Prominent Transcription Errors by Error-Type 



(Seg) (UnF) (IS) (TD) All 

Segmentation unfamiliar words Incorrect Spelling Totally distorted students 



Session 


Count 


%of 
transcripts 


Count 


%of 
transcripts 


Count 


%of 
transcripts 


Count 


% of Avg. per 
transcripts transcript 


1 


11 


92% 


12 


100% 


3 


25% 


6 


50% 


2.67 


2* 


10 


91% 


10 


91% 


6 


55% 


5 


45% 


2.82 


3 


7 


58% 


6 


50% 


1 


8% 


6 


50% 


1.67 


4 


1 


8% 


7 


58% 


4 


33% 


5 


42% 


1.42 


5 


2 


17% 


6 


50% 


0 


0% 


1 


8% 


0.75 


6 


5 


42% 


1 


8% 


2 


17% 


0 


0% 


0.67 



Note. Average per student. 1 1 transcripts 



Figure 1 1 presents the total count of prominent error types, without regard to their 
category, providing that each error type appeared at least three times in each 
transcription. It clearly shows a continual decrease in prominent errors and reflects an 
increase in student learning proficiency throughout the six sessions. 

Note that in the first two sessions, every student on average showed nearly three 
prominent error types, even though the first two sessions were relatively easy in terms of 
distortion level with the SNR not set correctly until the third session. The trend of 
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decrease in errors beginning in Session 3 suggests that students gained proficiency from 
the instruction they received and from listening to the distorted audio, even when the 
distortions levels were low. The phone line was an easy version; the static or babble 
levels were low and did not adequately cover the auditory message. 
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Figure 11. Average number of prominent error types per transcription at each session. 

Figure 1 1 demonstrates that the teaching was effective and the learning 
comprehensive. Even when the distortions became more difficult (Session 3), students 
still showed an impressive decrease in their prominent error types, because the instruction 
was tailored to each individual's needs, focused on the specific student's prominent error 
types. Results show that by the end of the research most students were able to overcome 
most prominent error types: There was an average decrease of 75%, from 2.67 to 0.67 



errors per transcript. 
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Different patterns of decrease in errors: "Error transfer and gradual mastering of 
tools" theory 

A comparison between Figure 10, presenting total error count of all types, and 
Figure 11, presenting prominent errors, shows that the major decrease in total errors 
(Session 5) occurred two sessions after the major decrease in prominent errors (Session 
3). The two trend lines are presented in Figure 12. 



Different Patterns of Improvement in Transcription Quality: 
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Figure 12. Two trend lines of error decrease. 



This phenomenon can be attributed to tailored teaching, the focus being on 
eliminating a number of prominent error types for each student, and can shed light on the 
hidden process of learning. In earlier sessions, as the learner was trying to cope with an 
average of three prominent error types, s/he could more efficiently focus on fewer types, 
most likely one at a time. The result of focusing on one specific error type at a time was 
that the count of errors related to the one specific error type decreased to fewer than 
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three, causing the elimination of that specific error type and leading to a decrease in the 
prominent error type count. However, although the count of prominent errors related to 
this specific type decreased, the count of other prominent error types increased. As a 
result of this "error transfer" from one type to another, the total error count remained 
stable. Only after four sessions did students gain proficiency and incorporate techniques 
to cope with more error types simultaneously without increasing their total error count. 
This "error transfer and gradual mastering of tools" theory can be supported by other 
indicators discussed in this paper. 

Prominent error type: Unfamiliar Words (UnF) 

At the end of each session, students were given the full transcription of the text 
they had heard (the solution) and were asked to indicate which words were unknown to 
them, not part of the vocabulary they already possessed. This information was cross- 
checked with student transcriptions to see how students coped with the challenge of 
transcribing a word that is not part of their active vocabulary. During the instruction 
sessions, students were provided with tailored instruction on how to improve their 
performance of writing unfamiliar words. Students were instructed to try to write the 
unfamiliar word as they heard it. They were instructed to use different methods in their 
transcriptions, such as grammar- and context-based techniques as well as the "write down 
the word to the best of your ability" method (Chapter 3), which included writing drills to 
master the new alphabet. 

Table 8 presents occurrences of the UnF prominent error type. Figure 13 
demonstrates how this error type occurrence decreased throughout the Main Study. 
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Figure 13. Percentage of appearances of unfamiliar word transcription errors. 



The UnF error type was the most common occurring error type in the first session: 
100% indicates that the UnF error type appeared prominently in every participant's 
transcript in the first session (Figure 14, Session 1). Over the course of the study, the 
UnF error type occurrence dropped to 8%, indicating that only one student still showed 
UnF as prominent by the time of the last session (Figure 14, Session 6). 
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Figure 14. Unf error type mapping: Performance breakdown by individual students. 
N/A: student did not participate in this session. 
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The pattern of decrease in this error type (Figure 13) shows three phases, 
indicated by two sharp drops in error occurrences: from the 90% area to the 50% area, 
and from the 50% area to the 10% area. Over the course of Sessions 3-5, half of the 
participants were challenged by the Unf error type (Figure 13 and Figure 14). Figure 15 
presents these "transit sessions," showing that students "alternated" between failing and 
succeeding in coping with this error type until they stabilized with a successful 
performance as shown on Figure 14, Session 6 (only one student showing this error type). 




Figure 15. Unf error type mapping: Performance breakdown, Sessions 3-5. 

Figure 15 demonstrates that the reported population of participants in the study 
behaved in a similar manner: All students modified their performance during Sessions 3- 
5 until they stabilized, as a population, on low Unf error occurrence. 
Prominent error type: Segmentation/de-segmentation. Segmentation error occurs when 
the transcriber breaks one word into two or more to transcribe it as multiple words. De- 
segmentation occurs when the transcriber runs two (or more) words together, so they 
appear as one word, especially when the speech is fast. Segmentation and de- 
segmentation represented common error types and their occurrences in the different 
sections are described in Table 8. Segmentation errors stem mainly from the fact that the 
non-proficient listener fails to identify where a single word starts or ends, resulting in the 
inability to separate two words from each other. The techniques to cope with these errors 



Ill 

involved speech characteristics and grammar-driven techniques. Figure 16 shows how 
the segmentation error presented throughout the study: 
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Figure 16. Percentage of appearances of Segmentation (Seg) transcription errors. 



The graph shows a significant and rapid drop in segmentation, indicating an 
improvement in proficiency immediately after the first two sessions. The rise in the last 
two sessions is not repeated in any other finding, and there is no indication that the 
difficulty of transcribed material was higher; however, it may be attributed to the 
decrease in the Totally Distorted (DT) error type, described in Figure 17. It is a valid 
assumption that, with the improvement in student proficiency, words that would have 
been marked as TD in previous sessions were now better identified yet still challenging. 
The result was a shift of error type: Words from error type TD that would have been 
illegible before shifted to the less severe error type of segmentation. 



112 

Prominent error type: Totally Distorted (TD) words. The Totally Distorted (TD) error 
type presents a high challenge for the transcriber. The error type occurrences count is 
described in Table 8. This error type occurred when students heard something but did not 
understand it at all. Students wrote down whatever they heard, even though it was 
completely illegible. This error type can occur due to one or both of two main reasons: 
(a) the student heard the word completely distorted or (b) the student had difficulty 
relaying what was heard on the audio onto the paper, due to incomplete control of the 
alphabet and letter sounds. Measures to address both possible reasons were taken, such 
as improving student hearing and assigning corrective homework. The homework 
included alphabet use and speaking/listening characteristics, such as "listen while 
reading" and "transcription puzzles" techniques. These activities demonstrated to 
students how different sounds came through the distortions. The graph on Figure 17 
shows the learners' responses. 
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Figure 17. Percentage of appearances of Totally Distorted (TD) transcription errors. 
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The TD error type appearance shows a pattern of student difficulties in coping 
with the most challenging parts of the audios, the most highly distorted words. Unlike 
with other prominent error types, students needed four sessions to improve their 
performance instead of two. This pattern is similar to the total error count discussed 
previously (Figure 10). Apparently, only after the students had mastered multiple 
transcription techniques could they cope with the most challenging parts. 

This error type appearance dropped rapidly as students gained proficiency in 
using methods to reduce it. In the last session, TD errors disappeared, which indicated 
that students could better hear words through the masking overcast and/or write them 
better on paper. As described before, some of the challenging words in the last sessions 
— instead of being marked as TD errors — may have been seen as less severe errors, such 
as segmentation errors, because they appeared to be more legible than TDs but still did 
not look like "correct" words. 

Prominent error type: Incorrect Spelling (IS) of known words. The Incorrect Spelling 
(IS) error type emerged from the challenge of using the new alphabet and correctly 
conveying the sound heard onto paper. When a student does hear a word but fails to 
transfer what s/he hears onto paper, especially in a foreign alphabet such as Hebrew, it 
reduces his/her transcription quality. However, not all spelling errors were counted 
during transcription analysis: Some spelling errors did not reduce the transcript quality 
and legibility because the sounds were correctly conveyed. For example, if the learner 
heard the word "ball" but wrote "bol," which sounds the same, it indicated merely a lack 
of spelling knowledge and exposure to the language rather than a problem with 
transcription or listening. The same applies to "knife" spelled as "nife" or "lamb" spelled 
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as "lam." However, if the learner wrote "Bole" or "Bowie" instead of "ball," or "nive" 
instead of "knife," (not the same sounds), then it was marked as a spelling error and 
addressed during the instruction. 

The IS error type occurrences in the different sessions are presented in Table 8. 
The researcher utilized multiple methods to improve students' production during the 
research; Most of them were practiced at home in the form of homework. The graph's 
irregular pattern displayed in Figure 18 can be connected to those HW activities, assigned 
before Sessions 3 and 5. Along with the pattern, the events of homework assignments are 
marked on the graph, to emphasize their profound influence on the exhibited learning 
pattern of students. 
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Figure 18. Percentage of appearances of Incorrect Spelling (IS) transcription errors. 

In the first session, this error type did not seem to be dominant. As the distortions 
were set to higher levels in Session 2, the count of IS errors increased. Therefore, the 
researcher assigned corrective HW to students between Sessions 2 and 3: transcription 
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voweling, letter recognition, and letter-sound related techniques. The graph shows that 
IS errors sharply decreased in Session 3, immediately after students worked on 
overcoming this type of challenge, despite the fact that Session 3 presented a greater 
auditory challenge — with the highest level of distortions — than the previous two sessions. 
Although the techniques seemed to be effective, the effect did not last and the count of 
this error type increased in Session 4, albeit to a lesser extent. Students were reassigned 
HW of the same nature; and, as a result, errors of this nature decreased to zero in Session 
5, only to increase again slightly in the sixth session. This learning pattern seems to be 
unique and different from the previously exhibited patterns: It shows a pattern of fading 
skill mastering, with the need to repeat and fortify skill acquisition until it becomes 
second nature. 
Coherency 

The Coherence/ Accuracy measurement was defined in this research as the native 
reader's ability to understand the transcriptions produced by the participants. The 
coherence measure was developed in this research to evaluate transcription quality. 
Following each session of teaching, in addition to a quality check based on errors, the 
researcher evaluated how coherent each transcription was to a native speaker depending 
on the number of words transcribed, their accuracy, and their role in the sentence. 
A numerical value rating the overall quality of each transcript was given. For more 
details, see the "Definitions" section in Chapter 1. The developed "coherency check 
protocol" is described under "Coherency/accuracy — numerical grading of transcriptions" 
in Chapter 3. 
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Coherency evaluation was found to be the most valid measurement of the 
quality of transcriptions. It produced the fastest results in follow-up sessions and it was 
the least time consuming to run. As a result, the Advanced Study that followed the 
described Main Study was primarily based on that measurement. 
Presentation of Data 

Table 9 presents the average scores that were given at each session to all the 
transcripts belonging to the same distortion type. For more details, see "Distribution of 
distortion types" in Chapter 3. 

Table 9 

Transcription Coherency Averages, All Distortion Types, Sessions 1-6. 



Session Phone distortion Babble distortion Static distortion 

1 83^0 733 79^0 

2 59.5 56.3 64.3 

3 86.3 45.5 40.0 

4 80.8 58.3 61.5 

5 81.3 67.5 86.5 

6 75.3 88.0 79.3 



Note. Average value per class per session. 

Static and Babble. Figure 19 displays the results of two distortion types that show 
the same trends: Both distortions show high results on the first session, a decline in 
the second and third sessions, and then a steady trend of incline with stabilization 
between 90-100%. 
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Static and Babble distortions, Sessions 1-6: 
Coherency Results and Learning Curve 
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Figure 19. Static and Babble distortions results. 



The results from the first two sessions was the rationale for tuning the difficulty 
level of the clips in Sessions 2 and 3. The feedback from the students showed that the 
SNR in the first session was too low — the distortion did not mask the main audio enough 
to pose a challenge — so the masking interference was amplified in the second session 
once, and again in the third session. The final setting of SNR was established in the third 
session, which yielded coherency results and feedback from the students indicating that 
the challenge was high enough. Sessions 3 and 5 posed an identical challenge of 
difficulty; they presented the same pattern of learning curve indicating improvement in 
students' listening and transcription skills. In Sessions 5 and 6, students observed that the 
distortions did not pose a high challenge any longer and stated on their feedback that they 
could easily pick up the target text through the background voice. One student wrote in 
Session 6, "The Static and Babble are getting a bit easier to listen through. Today's 
transcription was not so hard ..." 

In the same session, one of the most challenged students wrote: 
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While the distortion did change the voweling and pronunciation as per usual, I 
knew what it was supposed to be. I don't know if I'm improving or if I just know 
more. I don't know content wise, but I'm good. 



This feedback, coming from students with challenges, shows that a learning process took 
place and that students gained high proficiency much faster than expected. 
Phone Line. Figure 20 displays the results of phone line transcriptions: Phone Line 
transcription coherency showed high results in the first session. After the challenge was 
increased for the second session, the results declined but returned to a high coherency 
level on the third session, showing the rapid adaptation of the students to the distortions. 
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Figure 20. Phone line distortion results. 



In Sessions 1 and 2 students wrote, "After the first few minutes, forgot phone line 
was there" and "Easier than static, phone wasn't too bad." In Session 5 a student 
commented, "Phone remains easiest — I can at least hear all the words even though I may 
not know the word or how to spell it." 
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A complex phone line with a changing pitch was presented in Sessions 4-6, which 
yielded a stable result of high quality transcripts, indicating that a phone line is a 
distortion type to which students easily adapted. The only interesting result was that 
some students, as they reflected on Sessions 3 and 4 when the new-pitched distortion had 
just been introduced to them, commented that the distortion was hard and that some areas 
were not totally decipherable: "The distortion added to the phone made it more difficult. 
It went from the easiest to the most difficult. However, Static and Babble are not far 
from it." Another student wrote, "phone REALLY distorted whole words or parts of 
them and some I wasn't entirely familiar with (mostly spelling the words). It made it hard 
to transcribe especially because of the warping in the audio." A third student wrote about 
Session 6: "Oscillation of phone distortion made it difficult to pick out certain words." 
However, transcription quality and coherence results were still high — and did not agree 
with the students' perception of difficulty. 

Phone line distortion — an important voice feature in daily life experience — did 
not challenge the students, who demonstrated through their results that they could adapt 
quickly and flexibly. Phone line distortion was one of the more advanced distortions 
presented to students in the Advanced Study. 
Effectiveness of Teaching Materials 
Difficulty of distortions: Different learning curves 

The difficulty of distortions was estimated mainly through coherency results and 
student feedback. The first emerging trend from student feedback is that Phone Line 
distortion was the easiest to understand. For this reason, this distortion was made more 
difficult in Sessions 3-6. However, students rapidly adjusted to the new distortion, 
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which posed a higher difficulty level, and continued to report that this distortion posed 
the least challenge. 

The second emerging trend was that the research participants rated two 
distortions, Babble and Static, as more difficult than the Phone Line, but it was not clear 
from the results which distortion is the most difficult. It seems that students formed two 
groups: one that rated the Static as more difficult, and the other the Babble. However, 
there were transits of individuals between the groups throughout the sessions, and it is 
possible that the first distortion transcribed by a student in a session may be rated as the 
most difficult. One student wrote: "Babble harder. Since it was louder this time, and I 
transcribed it first, the others probably seemed easy." In the same session, another 
expressed another opinion: "Static harder — it blends with the words while others still 
provide some contrast." 

As the results are not conclusive, it is possible that there are two typical 
populations and that each is challenged by another characteristic. The researcher's initial 
assumption that the Babble background noise will challenge students more since it was 
expected to compete with the target text at a higher cognitive level (see theoretical 
background) was supported neither by the results nor by the student feedback. Students 
stated that because the babble noise was not legible at all and no words were picked up, 
they did not find it much harder than the Static to filter out. This led the researcher to 
create a more complex and challenging competing speech distortion — composed of 
legible speech in the target language — for the advanced phase of the research, as 
reported in this paper. 
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Measured results 

The presented data in this chapter shows improvement in the quality of 
transcription results throughout the six sessions of the Main Study. Students showed 
decreasing counts of errors and error types, along with increasing accuracy and 
coherency of transcripts. 
Students' opinions and feedback 

While engaged in the study, students were satisfied with their participation and 
achievements: Their feedback was positive. Students commented that the research 
contributes to their transcription ability, as well as to their listening proficiency. One 
student wrote: "You're pinpointing/polishing the skills of the students in their listening, 
and how to accurately annotate what they hear." 

When the Main Study ended, students were asked to anonymously provide 
extended feedback. All of them (a) supported such an instruction plan to be part of the 
curriculum in the future and (b) requested to continue in an advanced activity of the same 
nature; and some suggested ideas as to how to take the next steps. As a result, the 
researcher continued to an advanced phase, and students were introduced to additional 
distortions, such as competing speech and muffled audio. 

The following is from the anonymous feedback received at the end of the 
main study: 

Q: Would you recommend that future students go through the same kind of 

instructional experience? Why? 

• I would definitely recommend other students (especially newer classes) 
to undergo this instructional experience because it really conditions your 
hearing and helps you become accustomed to the different types of 
distortions. I distinctly remember getting transcription homework during 
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the beginning that was authentic material and it was much harder than 
this. I remember struggling and becoming frustrated, but with these 
sessions, learning how to selectively hear what I want to helped me 
overcome a lot of the listening obstacles I was experiencing. Having new 
students transition from non- authentic material to authentic material with 
these sessions would definitely help them with listening instead of 
jumping straight into audios that are too difficult for their level. 

• I would definitely recommend future students going through this type of 
instruction. I feel that it has sharpened or even increased my ability to do 
transcriptions. I may not know what the word means but I am very 
confident in my ability to transcribe correct "sounds." The letters may be 
off here and there but the "sound" should be correct. 

• YES. If a student has difficulty with listening THEY MUST find a way 
to cope with it, and to learn to listen to imperfect audio. Plus, if they can 
get better at listening to this material, and use the skills gained here, their 
overall listening will improve on regular listening for other things such as 
on the tests we undergo, and when they have to listen during their jobs at 
their duty stations. THIS is one of the most important skills you can 
impart to your students here other than the core of the language they are 
learning. If they apply this well, they can learn to listen better in English, 
Spanish, Hebrew, ANY LANGUAGE. THIS IS AN INVALUABLE 
SKILL. 

• "Yes, because it teaches you how to fine tune your listening, so that when 
your transcribing EEIs or listening to a passage you can focus your 
hearing to listen for buzzwords better". 

• "Absolutely. I can think of nothing that would hinder a student with this 
project. If anything, it will improve the student's ability to listen, 
transcribe, and [use] vocabulary". 



Summary of Findings, Main Study 

1 . Throughout the study, students 

a. stated that the distorted audio material was effective in training them 
to higher transcription proficiency; 
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b. stated that the instructional methods and strategies were effective in 
training them to higher transcription proficiency; 

c. increased their proficiency of listening; 

d. increased their proficiency of transcription; 

e. were satisfied with the research and the instructional plan; 

f. recommended that future students go through similar programs in 
Hebrew and in other languages; and 

g. commented that an early introduction of audio materials as presented 
in the study is effective and recommended for beginning students. See 
Chapter 1 under "Theoretical Foundations" for more details. 



Additional Findings 

1. Throughout the sessions, total error count dropped. 

2. Throughout the sessions, the number of four prominent error types dropped, 
or the error type was completely eliminated. 

3. Students demonstrated different patterns of error correction, error fading, and 
learning curves. 

4. Student indicators showed a decrease in the reported perception of stress and 
difficulty and an increase in student confidence, while experience and 
proficiency were gained throughout the sessions. These findings support 
previous works (CASL,2010; Vandergrift, 2006; Vandergrift, 2007; 
Vandergrift & Tafaghodtari, 2010; Vandergrift, 2010) and are discussed in 
Chapter 1 under "Theoretical Foundations." 
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5. Students showed diversity as individuals; however, analysis and class results 
outside the described research support the assumption that this group of 
participants is a homogenous group, a representative sample of the total 
population. 

Advanced Study 

As a direct result of students' positive feedback and emergent new ideas from the 
main research, the researcher initiated an advanced phase that immediately followed the 
Main Study. The focus of the Advanced Research was on describing how learners coped 
with exposure to more realistic and complex distortions developed by the researcher to 
resemble different electronic distortions common in the real communications world. The 
distortions represented three types of quality loss: 

(a) background noise that competes for listener attention and mental resources while 
processing the message (Competing Speech), (b) low-quality audio ("Muffled" 
distortion), and (c) loss of data and parts of words due to electromagnetic distortions 
("Chopped" distortion). All presented distortions included a basic and equal process of 
phone line and static noise additions, followed by a specific distortion process or addition 
of another interference, as described in the "Technical Procedures of Audio Clip 
Preparation" Appendix 3. Text verification was included, ensuring the texts to be at the 
same level. The Data Collection Sheets were modified after observations and feedback 
from the Main Study answered the research question. The detailed list of techniques used 
at each transcription was eliminated, and a structural change in the study was 
implemented to expose the whole class to the same distortion at the same session. As 
students were already experienced and proficient with the different transcription methods 
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and techniques, the focus of the instruction was on how fast proficiency could be 
achieved in transcribing each distortion and how best to do that. In most sessions, an 
optional pre-transcription activity (warm-up) was available to the students before 
transcription, and HW was assigned post-session to prepare for the next challenge. 
Presentation of Data 

Table 10 presents the average scores given at each session to the different 
transcripts of the different exposures. For more details, see "Distribution of distortion 
types" in Chapter 3, "Definitions" in Chapter 1, and "Technical Procedures of Audio Clip 
Preparation" in Appendix 3. 

Table 10 

Transcription Coherency Average, All Distortions Types, Sessions 7-18. 
Session of Competing Speech (CS) 

exposure distortion Muffled distortions Chopped distortion 

1 57 90 63 

2 70 60 89 

3 75 57 87 

4 86 74 

5 79 

Note. Average value per class per session. Sessions 7-10. 11-15. Sessions 16-18. 
Not conducted. 

Table 10 shows that not all distortions were used for an equal amount of 
exposures. The reason was that once a class reached proficiency, measured by a high 
evaluation of coherency and student response, the research concluded the interim goal of 
instruction for each distortion and advanced to the next one. 

Figure 21 is a graphic representation of all transcription activity conducted in the 
Advanced Study. It describes coherency, the most important measurement, and shows 
that by the last exposure of each distortion, students reach proficiency, with their average 
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scores stabilized, reflecting the high performance of participants. However, developing 
proficiency for each distortion required a different number of exposures. 
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Figure 21. Coherency results of all sessions, Advanced Study. 



The different patterns are the result of different side effects related to the way the 
research progressed; they will be discussed below. 
"Competing Speech" Distortion 

In the Main Study, the "Babble" distortion — designed to present the highest 
transcription challenge to students — was, in fact, as equally challenging to the students as 
the "Static" distortion, although theoretically competing at a higher thinking level. The 
students noted that the babble noise was not legible; it seemed to be simply masking 
interference rather than competing with the main message. Therefore, the researcher 
developed the Competing Speech distortion. For this distortion, the added competing 
speech was of radio news broadcasters reading temperature numbers. A woman's voice 
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was used on the first exposure, a man's voice on the second, and both voices on the third. 
The Competing Speech (CS) distortion was the first "realistic" distortion that students 
encountered, at their request. The technical procedures of preparation are detailed in 
"Technical Procedures and Audio-Clip Preparation," Appendix 3. 

The following feedback from the first exposure session are some of students' 
opinions regarding the effectiveness of the distortion on their learning: 

This was awesome! I could only hear through parts of it but still. . . It was stressful 
and really difficult and I would only hear vowel sounds at times but it was still 
fun! Definitely more aware of how I disseminate a passage. Used MOST of the 
techniques according to the feedback. 

Another student wrote: 



This is what I wanted to do from the beginning, Doron. This is the type of 
listening that I needed. I feel like I've had to work harder on this than any other 
listening so far. I don't know what all of it meant, but I think I heard most of it... 
The competing speech really made this more difficult. The other speech was 
actual speech instead of just babble. And it was constant, and the other speech 
was real enough (actual words) that you had to try not to listen to. The words 
were audible and audibly recognizable, so I had to tune them out. I feel mentally 
taxed because I had to consciously try to NOT hear them while listening to the 
target audio. The vocabulary and grammar was RIGHT AT THE LIMIT OF 
WHAT WE KNOW. . .this was good. . .difficult but good. 
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Coherency 

Figure 22 presents the results over the four sessions of exposure: 
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Figure 22. Competing Speech distortion coherency results, Advanced Study. 



The graph shows a continual improvement of transcript quality and legibility. 
The students' average score improved by 5-13 points at each session compared to the 
previous session: a steady learning curve. After four exposures (two weeks), the 
feedback from the students was that they had become proficient and that the CS distortion 
no longer presented a challenge. 

The written feedback elicited from the students reflected the same improvement 
in learning and proficiency. The following feedback quotations are from the same 
student, from the first to the last exposure: 

1. First exposure: "Very distracting. Hard enough to where I can't understand it 
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just because I can't hear it, very frustrating. The first one of these that I've been 
frustrated at." 

2. Second exposure: "Today's clip seemed difficult at first but the more you listen to it, it 
became easier but there were still words that I just couldn't get because of the 
distortion. Overall it wasn't bad." 

3. Third exposure: "The homework from last night, listening to the weather, was helpful 
in the fact that it trains you to tune out things, I think that helped but it was very 
difficult to hear so it was frustrating not being able to understand half of it but overall I 
liked it." 

4. Last exposure: "Today's clip was not very difficult, I am starting to not notice the 
noise, and just focus on the text, I don't even have to pay attention to blocking it out 
now, my brain just kind of expects it now, there are always certain words that I don't 
get and will not be able to get just because I cannot hear them but I feel like overall I 
am improving a lot." 
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Figure 23. Coherency grades sorted by value at each exposure, and the improvement of 
average by percent. 
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Figure 23 presents the participants' Coherency grades, sorted by their value. 
The improvement trend is clear. Six students, half the class, received a grade under 50 on 
the first exposure, two students received a grade under 50 on the second exposure, and 
only one student on the third and fourth. 
Transcription sample 

The following example on Figure 24 is showing the transcriptions of the same 
student, only three days apart. Between the first and second exposures, the student's 
transcription quality improved strikingly, from a 46% score to 93%. 
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Figure 24. Comparison between the productions of the same student: First exposure (left) 
vs. second exposure (right). 



Figure 24 shows that on the first transcript the student left whole areas blank, as 
she could not figure out all the text. The next transcript is complete and the sentence 
quality is much higher, as indicated by the scoring on the right side. In addition, the 
student's perceptions of the difficulty of the task and her confidence in her work 
improved by two levels, and the student's perception of her stress lowered by two levels. 
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Student indicators 

Similarly to the Main Research, students were asked about their perception of 
each session. Table 1 1 shows the average results of each session. 

Table 11 

Student indicators, Competing Speech transcriptions. 



Session of exposure 


Difficulty 


Stress 


Confidence 


1 


4.6 


4.4 


1.6 


2 


3.6 


3.1 


2.3 


3 


3.4 


2.7 


2.5 


4 


2.2 


1.8 


3.8 



Note. Average value of class in each session. 

Figure 25 is a graphic representation of students' average perceptions over four 
sessions: The trend lines show clear decreases in perceptions of stress and difficulty and, 
at the same time, increases in perceptions of confidence. 
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Figure 25. Competing Speech, students' perception indicators, Advanced Study. 
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Importance of pre-transcription warm-up 

Warm-up activity was available to the students in most sessions prior to 
transcription, in order to prepare them for the effort. The activity consisted of previous 
clips accompanied by their transcripts and a "Transcription Puzzle" activity, which was 
found to be helpful with challenging auditory materials. Table 12 presents students who 
did or did not perform a warm-up activity and the change in their grades from one 
exposure to the next one. 



Table 12 

Connecting students' pre-transcription warm-up with change in their grades. 



Students' reported use of warm-up 


Did warm-up 


Did not warm-up 


Change (points) 47 31 25 23 18 15 5 4 -6 


4 2-2 



Table 12 correlates the students' reported use of warm-up before Exposure 2 and 
the change of coherency grades: Grades of students who did not warm up remained 
stable, while grades of students who did warm up showed improvement. On their 
feedback, students referred to the warm-up or homework aspect: "HW efficient. 
Becoming accustomed. Suggest harder warm-up." Another student commented: 

For warm-up I chose to listen to a few of the previous sessions. I decided to 
listen to a couple of the previous sessions a few times through and then listen to 
them again while reading the solution. It was to get my mind ready and 
"Hebrew'ized" before listening to today's session. 
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A third student wrote: 



Like last time, I decided to listen to the previous session's audio and listened to 
each narrative separately, then the competing speech together. This helps me 
know what to isolate when I listen, and to know what I'm looking for when I play 
it back. I wasn't entirely familiar with the undertone competing speech so it was 
a good refresher for words and different voice tones. I think the warm-up really 
helped me because it sounded as if the undertones were very similar in terms of 
the tone of the narrator's voice (very low and deep). It helped me remember 
which sounds to listen for and helped me prepare in isolating the voices. 

The Monday effect 

Following student feedback, the Monday session of the research was moved to 
Tuesday. Student feedback suggested that administration of the research on the first 
hours after the weekend impedes their ability to focus. This feedback was in agreement 
with the moderate improvement of grades in the third session of only five points. As a 
result, the next sessions did not immediately follow a long break or weekend. 
Student perception regarding the CS distortion effectiveness 

The competing speech distortion was the first complex distortion, with a 
combination of phone line, static and another auditory manipulations. For the 
participating students, this was the first encounter with a "realistic" distortion, and the 
students' impressions are important in assessing how effective and authentic the 
distortions sound. The following are students' feedback: 

• I think this research is very productive and useful from a language-learning 
standpoint. I haven't noticed the changes because it is gradual, but looking at 
the results I realize that I am better than I have ever been at listening and I 
continue to do better with more practice. The research is a good tool to 
continue to better the listening proficiency of students in language courses. It 
should continue. 
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• I would like to see the research continue. Honestly, it is benefiting not only 
you and your research but obviously the student body as well. I think that all 
students at DLI should go through a course just like this. I think in 3rd 
semester when students are on the maximal level, they should still be doing 
this. I also think that in the end of first semester it is vital because it will 
prime them for what is to come. It is simply our job to hear through noise. 
And until now, we have not done that. No student at DLI has been given to 
opportunity to do his or her job in the training environment. I don't care how 
well you are at the language, if you can't hear through sound or distortion you 
won't be able to hear what you need. I am truly thankful for the opportunity 
to better myself and my country. This is what I need to do in the long run. . . 
and this process is helping in every way to reach my goal. 

• I feel like it is especially helpful because I could feel my mind tuning out the 
background even more and just kind of absorbing what it is hearing. It felt 
more like I was listening to my own language maybe at home while other 
things were going on instead of normal foreign language listening. 

Overall, students thought that the newly implemented research/instructional plan was 
effective. Students recognized its contribution to their proficiency and asked to continue 
the activity. 
"Muffled" Distortion 

The Muffled distortion is a process that uses a Low-Pass filter in the audio 
processing software to muffle the vocal message. It sounds similar to what a listener on 
the phone would hear if the person on the other side would cover the phone with his palm 
or would carry the phone in his pocket. Technical and procedural instructions regarding 
preparation of the clip can be found in the "Technical Procedures and Audio-clip 
preparation" appendix. The first exposure to this process type was set at a level that was 
too easy for the students; as a result, the audio was transcribed with no challenge to the 
students. To calibrate the next clip to a setting that would challenge the students, the 
researcher was helped by the best listener in class to set the audio processing program to 



135 



a challenging level. The result was that, as expected, the second exposure produced 
transcripts of lower quality, which was a good starting point for improvement. 
Coherency 
Presentation of Data 

The average quality score at each session of the exposure to the "Muffled" 
distortion is presented in Table 10. Table 13 presents the changes in grades throughout 
the exposure sessions. 

Table 13 

"Muffle " Distortion: Transcription Coherency, Averages and changes in grades. 

Session of Coherency score Change from previous Change from previous 

exposure J . . , . B * „ , 

exposure [points] exposure [%] 

1 90 ~ ~ 

2 60 -30 -33% 

3 57 -3 -5% 

4 74 +17 30% 

5 79 +5 7% 

Note. Average value per class per session. 

Figure 26 presents the results of the five exposures. Note that all texts were 
checked to be of the same difficulty level. This exposure's "run" was the longest in the 
Advanced Study, lasting five sessions, because (a) the first setting was too easy so the 
researcher initiated a "calibration process" for difficulty level setting and (b) the 
challenge of the distortion after calibration yielded another pattern of learning curve. 
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Figure 26. "Muffled" distortion coherency results, Advanced Study. 

Figure 26 demonstrates that the actual learning curve started only at the second 
exposure. Because the distortion presented a higher challenge to the learners, two 
exposures were required in which the students produced low-quality transcripts in order 
to break down the barrier to producing more coherent transcripts. When that eventually 
happened, the first improvement was considerable — 17 points (33% increase), as shown 
in Table 13. The fifth exposure showed a smaller improvement — 5 points (7% increase), 
which brought the students to a good coherency level and started a steady performance on 
the graph. 
Student indicators 

Table 14 presents the average score of coherence, as well as the average values 
for student perceptions at each session of the Muffled distortion transcriptions. Note that 
the first exposure should not count as part of the learning curve, as it was "too easy." 
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Table 14 

Student indicators, "Muffled" transcriptions. 



Session of exposure 


Score 


Difficulty 


Stress 


Confidence 


1 


90 


3.20 


2.10 


2.90 


2 


60 


4.45 


3.36 


2.00 


3 


57 


4.45 


3.27 


1.54 


4 


74 


3.45 


3.00 


2.45 


5 


79 


3.50 


2.30 


2.60 



Note. Average value per class per session. 

Figure 27 is a visual representation of student perceptions throughout the study. 
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Figure 27. "Muffled" distortion, Students' Perception Indicators, Advanced Study. 



The graphs show a strong correlation between high levels of difficulty to 
transcribe and high levels of stress with low levels of confidence, on exposures 2 and 3, 
when the coherency scores were low. During the last two sessions, as quality improved, 
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student indicators stabilized: Stress lowered, the perception of difficulty lowered, and 
confidence increased. 
"Chopped" Distortion 

"Chopped" distortion was developed to represent low quality audio resulting 
from loss of data/parts of words during electromagnetic distortion occurrences. This was 
the shortest set of exposures, as students showed immediate improvement after the first 
exposure, then maintained a stable performance. 
Coherency 
Presentation of Data 

Table 15 presents scores and student indicators: 



Table 15 

Student indicators, "Chopped" transcriptions. 



Session of exposure 


Score 


Difficulty 


Stress 


Confidence 


1 


63 


4.0 


2.5 


2.4 


2 


89 


2.5 


1.8 


3.3 


3 


87 


2.5 


1.3 


3.5 



Note. Average value of class per session. 



Figure 28 shows low quality of student transcripts, as expected with a new 
distortion adequately set to challenge the students. 
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Figure 28. "Chopped" distortion coherency results, Advanced Study. 



The unexpected result in this exposure set is the short time that it took students to 
become proficient: As early as the second session their average coherency was excellent, 
having increased by 26 points (41%) from the first session. At the next session the 
students' graph line reached a plateau and stabilized. This fast recovery can suggest that 
students became proficient at using transcription techniques and showed an acquired 
listening and performance flexibility, which helped them leap between distortion types to 
adapt faster as the research progressed. 
Student indicators 

Student indicators are presented in Table 15 and Figure 29. The connection 
between students' perception and the actual quality of the transcriptions is clear: The 
longer the exposure to the new distortion, the higher the increase in coherency and 
confidence and the lower the stress and perception of difficulty. 



140 



a* 
OJD 

a 

es 

u 

o 



5.0 



4.0 



3.0 



2.0 



.Si 1.0 
-« 
S 



0.0 



Student Indicators, Chopped distortion, Advanced Study 
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Figure 29. "Chopped" distortion, students' perception indicators, Advanced Study. 



Correlations Found in the Advanced Study 

Measured student indicators are important in determining the transcription 
quality, and they interrelate with each other. The independent factor is the perceived 
challenge the distorted audio poses to the student. Between the source and the end result 
quality are the students' stress and confidence, which can be influenced by the teaching 
program. The outcome, coherency of the transcription, is the independent variable. 

Using the Pearson correlation coefficient, strong correlations can be shown 
among all student indicators and students' measured quality of transcriptions 
(Coherency). The examples below are not statistically valid because the sample size is 
not big enough; however, the trend is clear and can be observed in the following figures. 

Figure 30 demonstrates a strong positive correlation between student confidence 
and transcription quality. This correlation shows that a confident student will usually 
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produce a higher-quality transcription, and vice-versa: Positive past experience with high 
results will increase student confidence. 
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Figure 30. Correlation between transcription quality (coherency) and confidence 

The combined effect of encouraging results from a previous session and a more 
confident student entering the next session produces a positive cycle that enhances 
student performance. The high correlation (R = 0.82) shown in Figure 30 emphasizes 
the importance of building student confidence. 

Figure 3 1 shows another strong positive correlation between difficulty of 
translation and transcriber's stress. Stress is a bodily reaction of the student that can 
relate to perceived difficulty of the task. 
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Figure 31. Correlation between Students' stress and difficulty of transcription 

Figure 31 and Figure 32 reflect that when perceived challenge is high, perceived 
stress is high, contributing to a lower quality of transcription. 
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Figure 32. Correlation between transcription quality (coherency) and stress 
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Figure 32 demonstrates that coherency has a strong negative correlation with 
student stress: Stress can affect the quality of the final result. On the other hand, higher 
confidence in one's own ability can reduce stress (Figure 33). 
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Figure 33. Correlation between students' stress and confidence 

Even though the task is challenging, high confidence in one's performance 
(related to previous success, added experience) or stress reduction (cognitively reducing 
self-induced stress) can break the cycle that limits performance, thus leading to a 
successful cycle of high confidence/low stress/quality performance. 
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Summary of Findings, Advanced Study 

1. Students were satisfied with the realistic distortions and thought the 
distortions helped raise their transcription proficiency. 

2. Students thought the instructional plan helped them meet the challenge of the 
distortions. 

3. While reaching proficiency, students showed different learning curves. 

4. With all distortions, students reached proficiency within two to four 
exposures. 

5. Students showed high correlation between their indicators and their 
proficiency. 

6. Homework and pre-transcription task activities helped students reach 
proficiency. 
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CHAPTER FIVE: DISCUSSION, IMPLICATIONS AND RECOMMENDATIONS 

Overview 

The purpose of the study was to develop level-appropriate imperfect audio 
materials for early stage learners of Hebrew and to evaluate the efficiency of the 
instructional plan to improve students' ability to cope with these materials. It was 
a mixed-methods study: The qualitative part was derived from the students' weekly 
evaluations, the quantitative part from the numerical values extracted from student 
transcriptions and feedback. 

Teaching linguists to proficiently transcribe imperfect audio materials can be 
improved if the student is exposed to such audio early in the course of study. In practice, 
the teaching challenge is the low availability of both instructional materials and 
methods appropriate for the early levels of the language course. This research 
approaches the issue in a practical manner by (a) offering a technique to produce 
unlimited quantities of curriculum-appropriate and level-appropriate realistic auditory 
materials and (b) offering an instructional plan to effectively use these auditory materials 
to bring students to proficiency. The participants in this research were adult students in 
the early stages of learning Hebrew; however, the presented study is applicable to all 
foreign language learners. 

The purpose of this study was to evaluate the effectiveness of both the developed 
instructional audio materials and the developed instructional plan, aimed to increase 
student proficiency in transcribing distorted materials. The researcher generated audio 
for the purpose of training. The materials had to be level appropriate and also had to 
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simulate realistic communication distortions. Subsequent to the exposure of learners to 
the imperfect audio materials, the study evaluated the effectiveness of the teaching plan 
and instructional methods to cope with the challenge of transcribing distorted audios. 

The study reports student perceptions of the instructional program as well as their 
success in the transcription tasks. The report incorporates qualitative data collected from 
the participants' feedback and the researcher's evaluation of transcription quality. 
Presented findings are tentative, due to small sample size. 

Discussion of Findings 

This research was triggered by the challenge faced by the Defense Language 
Institute (DLI) to increase proficiency of its graduates and to advance greater success 
with distorted audio. The obvious need to introduce future linguists to such 
materials early in their learning process had been constrained because distorted audio is 
(a) characteristic of authentic material, (b) not language controlled and, (c) in most cases, 
not appropriate for teaching at the beginner level when the learner is still challenged by 
the language. 

The findings show high satisfaction with both the teaching program and the 
produced instructional auditory materials. Participants recognized the importance of the 
instructional plan and its contribution to their transcription proficiency. The participants 
showed higher flexibility than expected. They adapted to new distortion readily, and they 
presented shorter and more rapid learning curves as the research progressed. At the end 
of the research, participants were much more proficient than at the beginning, suggesting 
that the instructional plan and audio materials were effective. 
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Additional findings show that students who tend to use multiple listening and 
transcription methods while combining different strategies (Bottom-Up, Top-Down, 
grammar related, metacognition) improve their performance more than others. Also, an 
analysis using the Pearson correlation coefficient was used to confirm the existence and 
strength of the relationship (R ranged between 0.74 to 0.84) between student perceptions 
of confidence, stress, and difficulty of task and the quality of transcription, which reflects 
student proficiency. These findings show two cycles: the negative cycle of high 
challenge leading to greater stress and inducing lower performance; and the positive 
cycle (where a good teaching plan should aim) of awareness, experience, and positive 
past results increasing self-confidence, engendering less stress, and promoting higher 
performance results. 

The perceptual and numerical results of the research suggest that the presented 
teaching program, composed of schoolhouse-generated distorted audio and instructional 
methods, can be applied in similar teaching environments to the one in which the study 
was conducted. As the program is not language specific, it can be applied to other 
languages in the future. The imperfect audio material can be adapted to represent more 
distortions, as required. Most of the listening and transcription techniques can be used 
for many languages; some, which are language specific, may need slight modifications 
and minor changes to tailor them to specific languages. In addition to confirming the 
efficiency of the audio materials and the teaching plan, the study shows that (a) early 
exposure of students is effective, (b) students developed flexibility in adapting to 
different distortions, and (c) students were satisfied with the overall research. 
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Conclusions 

Research Question 

The research question stated, "Which teaching materials and teaching techniques 
appear to be most effective in helping learners acquire the ability to cope with low- 
quality audio?" 

Conclusion 1 (RQ part A): Teaching Materials Effectiveness 

RQ answer: All teaching materials were effective; each clip type played a role at a 
different point in the study and each helped students build proficiency to the next level. 

Effectiveness is when the students are challenged at the first exposure and then 
show an improvement in the following sessions. In Chapter 4, the study presented 
multiple learning curves showing improvement in connection to all distortion types. 
Multiple statements elicited from the students indicate that the distorted audio material 
was both challenging and beneficial to their increasing proficiency. 

The gradual exposure of students to distortions, from simple, one-distortion type 
clips to more complex types, was an efficient structural way to bring the students to 
proficiency. At the same time, students were building basic skills of listening and 
transcription strategies. 

1 . The simple, one-distortion clips used in the Main Study were effective in the 
beginning, and they served to create a solid proficiency base for the next step, the 
more complex (Advanced Study) distortions. The students gained experience in 
Static, Babble, and Phone Line during the Main Study; they acquired proficiency in 
using transcription methods and strategies through using these less-complex audio 
distortions. Exposure to the three simple distortions was done simultaneously: At 
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each session students were transcribing and focusing on one specific distortion while 
being exposed to the other two. 
2. The Advanced Study clips involved multiple distortions, which made them more 
realistic. Phone Line and Static were components of the "basic" clips, while the 
specific distortion was a "rider" that defined the actual distortion type. By the time 
the students had to cope with these distortions, they had become skillful in using 
transcription and listening strategies, and they had adapted their hearing to the 
masking background. Thus, they were able to cope with the more complex clips. 
Focusing on the single clip properties, the most challenging clips were the clips 
with the lowest Signal-Voice Ratio (SNR), or the most aggressive distortion, as 
transcribed text was masked by the background or distorted to the level at which the 
student had to get out of his/her comfort zone as far as possible. The "Muffled" 
distortion (Figure 26) exemplifies the phenomenon: The rigorous distortion demanded 
two recovery sessions for the students to break through the barrier (Figure 26, exposure 2, 
3) to achieve proficiency. However, a high SNR, when students indicated the clip was 
easy, did help students gain some experience, although it was less effective than the 
lower SNR. 

The instructional distorted audio materials are most effective when they are 
combined with as many transcription methods and strategies as possible. For example, 
text that contains grammar and context is more effective than single-word clips that 
would not allow for transcription strategies to be used. 
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Conclusion 2 (RQ part B): Teaching Methods Effectiveness 

RQ answer: All teaching methods were effective and helped students increase 
proficiency. The best results were achieved when students (a) used multiple methods of 
different approaches, such as Top-Down, Bottom-Up, Grammar-Driven; (b) combined 
different learning skills (visual, auditory, kinesthetic); and (c) employed cognitive skills. 

Effectiveness can be assessed by extensiveness of use and correlation to 
other factors. In addition, different phases of the study can show changes in trends 
of effectiveness. 
Extensiveness of use 

Figure 5 shows the prominent techniques used in the Main Study: familiarity with 
vocabulary (Bottom-Up); understanding the theme (Top-Down); grammatical agreements 
(grammar-based); and Repeated Listening (Top-Down and Bottom-Up, depending on the 
students' challenge and use). Figure 4 shows an increase in use of methods, which 
correlates with Figure 19, demonstrating an increase in the quality of transcription as the 
Main Research advanced. Students continued to attest to increased proficiency and 
increased use of methods. 

The Advanced Study feedback and results represent a later phase in student 
proficiency. This phase is characterized by more challenging listening materials and a 
more proficient transcriber. In this phase, use of the following transcription techniques 
increased, especially as support before and after the actual transcription sessions: 
repeated listening, deferred listening, "write-the-best-you-can while transcribing, 
Transcription Puzzle, and listening while reading. By repeating the transcription of clips 
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they were already exposed to, before and after the actual transcription sessions, students 
gained proficiency that translated to higher success on the new challenge of the next first- 
time exposure. 
Student indicators 

In both the Main and Advanced studies, students showed decreasing stress, 
decreasing difficulty perception, and increasing confidence, which correlates with the 
rising use of methods extensiveness. Figure 9 and Figure 4 demonstrate this pattern 
similarity. At least some of the drop in stress, the rise in confidence, and the decrease in 
perception of difficulty can be attributed to the effectiveness of methods used. 
Transcription errors 

Error count deceased dramatically throughout the Main Study. Figure 10, Figure 
9, and Figure 4 demonstrate this pattern similarity. At least some of the contribution to 
the count drop can be attributed to the effectiveness of methods used. 

The total count of prominent errors decreased dramatically throughout the Main 
Study, as demonstrated in Figure 11, which correlates with Figure 4. Use of some 
specific teaching methods can be attributed to specific error type decreases more 
than others: 

1 . The Unfamiliar Word error type decreased with method use related to mastering of 
the new alphabet and conveying the sounds onto paper (Figure 13). Students were 
instructed to write the unfamiliar word as they hear it and to incorporate other 
methods in their transcriptions, such as grammar-based and context-based techniques, 
as well as the "write down the word to the best of your ability" method. 
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2. Segmentation errors (Figure 16) were addressed using techniques related to 
speech characteristics and grammar-driven techniques. 

3. Totally Distorted error types (Figure 17) were eliminated by addressing student 
hearing and assigning corrective homework related to alphabet use and 
speaking/listening characteristics, such as "listen while reading" and "transcription 
puzzles" techniques. In these activities, students learned how differently they heard 
sounds through the distortions. 

4. The Incorrect Spelling error type was successfully addressed by corrective homework 
that utilized transcription voweling, letter recognition, and letter sound exercises. 
Figure 18 demonstrated the strong tie between the assignment of HW and the 
decrease of errors. 

Conclusion 3: Acquired Performance/Listening Flexibility 

Students acquired a high level of listening flexibility and performance ability. 
The participants' ability to adapt to different types and levels of distortion increased, and 
their ability to adapt to more advanced distortions at a faster pace was demonstrated as 
the research progressed. The fast recovery on Figure 28, which described a difficult 
distortion that eliminated data needed for the transcription, compared with slower 
recoveries on Figure 26, Figure 22, and Figure 19, suggests that such flexibility allowed 
students to "leap" across various types of information distortion: (a) energy or content 
masking in Static, Babble, and Competing Speech; (b) change of frequency in Phone 
Line and Muffling; and (c) elimination of information in the "Chopped" distortion. The 
"leap" phenomenon can also be attributed to the increased skill and higher proficiency 
acquired by using transcription strategies. 
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Conclusion 4: Error-Focused Instruction Leads to Error Reduction 

With error-tailored instructions, general error count and specific error types can 
be reduced, resulting in transcription quality improvement. 
Conclusion 5: Early Introduction of Learners to Distorted Audio 

The early introduction of distorted audio to adult learners of Hebrew as a foreign 
language is efficient and beneficial. In this study, students were first introduced to the 
research during their 13 th week of the 47-week period. Because the content of the audio 
was level appropriate, the learners were not challenged by content and thus were able to 
concentrate on overcoming the distortion. In their feedbacks, students said that they 
enjoyed going through the instruction; they did not say it was challenging because of 
their course level. There was no information in the findings to indicate otherwise. 
Conclusion 6: Positive and Negative Cycles of Learning 

High correlation between student indicators and quality of transcription reveals 
the positive cycle of learning that should be applied to a successful instruction plan as 
opposed to the negative. The positive cycle — awareness of the process, experience, 
and positive past results — increases self confidence, reduces stress, and produces 
higher quality results whereas the negative path — related to stress produces lower 
quality results. 

Conclusion 7: Student Satisfaction 

On their written and verbal feedbacks, students expressed that they enjoyed the 
instruction program and recognized its contribution to their listening and transcription 
proficiency, to their course test results, and to other lingual and learning-related fields 
(i.e., the reinforcement of grammar and self-awareness in the learning process). All 
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students recommended incorporating the study in future courses. All students requested 
to participate in similar activities after the research in order to maintain their proficiency. 

Implications 

This research offers applicable solutions to challenges currently faced by faculty 
and developers. The research defines an instructional plan that (a) offers a way to 
produce an unlimited amount of level-appropriate auditory teaching materials, which can 
allow the student to cope with distortions without being challenged by content, and (2) 
offers level-appropriate teaching strategies designed to cope with distorted audio in the 
early stages of the Basic Program. 

The implication of this research is that, if widely applied, it will allow students to 
be taught how to listen, starting in the very early stages of the Basic Program course. 
Some prominent additional factors are (a) the suggested instructional plan can be applied, 
with minor changes, to all languages; (b) the plan has zero cost and can be implemented 
using existing resources at the DLI; (c) application of the plan is very simple and 
production of materials can be done by an individual teacher, team, or department; (d) the 
plan uses existing curricular materials, eliminating the need to validate and verify the 
content of the teaching auditory materials; (e) the plan is highly flexible and can be 
adapted to new types of distortions or to future advancements in teaching and technology. 

The combination of these factors and the encouraging results presented in this 
paper, which is essentially an experimental pilot program, suggest that future linguists 
will be better prepared for their future service assignments of voice interceptors in 
the field. 
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Recommendations for the Profession 

This study may help language teachers to efficiently approach the teaching of 
low-quality and other types of challenging audio. The recommendations have two areas 
of focus: teaching of the language profession and teaching of listening. By their very 
nature, the recommendations are intrinsically entwined. 

Early and careful introduction of any challenging material is always 
recommended. When a student is exposed to a challenge early in the process of learning, 
s/he is given more time to (a) process the challenge and defeat it and (b) form productive 
learning habits. In addition, setting standards at the beginning reduces resistance later on. 
The proposed audio preparation instructions allow the teacher to control the content of 
the audio and its distortion level. An early use of teaching techniques and strategies can 
help the teacher instill productive and flexible use of methods in his students in the early 
stages, thereby increasing their adaptability to different distortions. 

Combination of teaching strategies and learning style. The nature of the 
suggested techniques is diverse — based on grammar, world knowledge, and vocabulary 
acquisition. Techniques range from the sub-word level (grammar, conjugations, and 
other derivatives of the word) with high cognitive requirements, to the bottom-up word 
and sub-sentence level (grammar, agreements of words and sentence parts) to top-down 
techniques that require world knowledge and high cognition of the adult learner. In 
addition, such methods use different diverse learning styles — visual, auditory, 
kinesthetic — and can be tailored to different learners. The ability to master these 
techniques can benefit any learner of any language at any skill level, thereby yielding a 
successful learner who can (a) stagger from one method and strategy to another, (b) 
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triangulate information derived from different sources within the audio/text, and (c) 
overcome the challenge. 

Andragogy, the tailoring of teaching to the cognitive level of the adult learner, 
who may be less challenged and can get bored by simple subject matter in the early 
stages of learning, is always recommended. When a teacher explains learning strategies 
to the student, a powerful and precious byproduct is produced: the increase of 
metacognition within the student. This can be leveraged to increase the student's interest 
in learning, which in turn increases his or her self-esteem. As a result, the student can 
"feel the wheels turning in his head," even in the early stages of language study, and go 
beyond vocabulary memorization and the use of simple sentences typical to this 
learning phase. 

Build a long-term teaching plan. This paper describes a research that was carried 
for a limited time. However, in order to maintain the acquired proficiency, the teacher 
must continue challenging the students. One possible plan would be two hours of weekly 
exposure to the suggested program, followed by a weekly hour of similar challenge that 
would increase in difficulty according to the advancement level of the class. 

Recommendations for Future Research 

Recommendation 1: validation of results. In order to validate the results of 
the study, it should be repeated with more participants. This will provide statistical 
validity to the numerical components of the study, which offers a plethora of 
numeric information. 

Recommendation 2: validation for more languages. In order to validate the 
assumption that the suggested plan is indeed applicable to languages other than Hebrew, 
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research should be conducted in more languages. Some adaptation to language- specific 
teaching methods should be made, such as focusing on other characteristics of the target 
language, changing grammar-related or speaking-related methods. 

Recommendation 3: earlier introduction. This study was to be initiated during 
the first trimester (between the 4th and 6th weeks of a 47-weeks course). However, due 
to technicalities it started in the 13th week. Although results are encouraging and show 
effectiveness of the teaching plan and its components, an earlier start should be 
considered, bearing in mind that the obstacle of level-appropriateness of audio clips has 
been removed by the study. 

Recommendation 4: use of more distortions. The participants in this research 
demonstrated acquired flexibility and proficiency. It is recommended to try more 
distortions of different types: energy masking, change of frequency, elimination of 
information, etc. 

Recommendation 5: realistic distortions. One important goal of the researcher 
was to enhance student exposure to realistic distortions that resemble the challenges they 
may face in the future. Due to Operational Security limitations, the researcher was not 
able to obtain authentic examples of such field materials and thus generated distortions 
based on his best judgment and experience as a former soldier and his skills in 
technology and electronics. It is recommended that the distortions used in this research 
be compared with real auditory materials and fine-tuned accordingly. 

Recommendation 6: utilize students to set distortion levels. In order for the 
students to go as far as possible out of their comfort zone and be exposed to challenging 
distortions, set the distortion level with a high-performing student before introducing it to 
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the class. Ask the student to point to the level where s/he is on the verge of not 
understanding the message; make that the distortion setting for the "+1 target level" 
approach, as students will be properly challenged with material that is just slightly above 
their level. 

Recommended future studies 

One possible future study can be performed in another language with a larger 
number of participants. Pre-research introduction of transcription techniques is advised, 
to reduce the adaption time of students. The research should use the same basic 
distortions at the beginning (phone line, static, and babble), introduced in separate audio 
clips in the same teaching session as it was performed in this study, to create a gradual 
exposure to the distortions. It is recommended to identify one high-performing student 
who will help calibrate the difficulty level of the audio clip before the first introduction of 
each distortion type. 

Another possible future study, focused on DLI students, would encompass a 
larger number of distortions, as realistic as possible. It would build on the above- 
recommended study, with more advanced realistic distortions as similar as possible to 
what the linguists will face in their future assignments. 

A third proposed study is focused on statistic validation of the results. It requires 
another methodological structure that will allow the comparison of two groups of 
learners: a control group who will study current curriculum and transcribe the curricular 
audio and a study group who will transcribe an imperfect curricular audio. Because 
numbers of participants should be significantly larger, it is recommended to perform this 
suggested research in a language with a sufficient amount of students. Different teacher 



159 

effects should be considered and compensated for, possibly by assigning one specific 
teacher for all instructions of both groups. 

Summary and Concluding Statement 
This study was conducted by a researcher who functions on a daily basis as both a 
frontline teacher and a developer of teaching material, who is at the crossroads of many 
stakeholders: the student, the teacher, other academicians, the management, and the client 
(i.e., the next service place of the learner on the next service assignment). As such, this 
research is offering a practical and applicable tool — to the system, to teachers and 
learners — a tool coming from true operational need: attaining listening proficiency to 
imperfect audio. 

The study focused on the applicability and efficiency of a proposed instructional 
plan involving the development of teaching materials, the development of a program to 
teach these materials, and the assessment of results. Results showed plan efficiency and 
applicability, as well as satisfaction of learner. In addition, the plan is highly cost- 
effective, considering the fact that the plan utilizes existing curricular materials and 
existing resources of the system, requiring only minimal professional training for the 
faculty or developers. 

The value of the study is in offering practicality in the teaching of a challenging 
aspect of language acquisition to professionals who will have to perform under 
challenging conditions in the future. We now know it is possible to begin exposure to 
distorted audio very early in the learning process, as soon as the material is level 
appropriate; in addition, we know how to remove level-appropriateness obstacles. We 
also know how to better cope with teaching challenges and how to improve teaching 



160 

techniques. As the study is paving the beginning of the road, these results suggest 
consideration of further study for the purpose of future improvements and future 
application of the plan to more students in more languages. 
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APPENDIX 1: The Interagency Language Roundtable (ILR) Scale 

(Language Proficiency Assessment) 
Based on ILR (201 1), the ILR listening scale is presented below. 

Description 

The Interagency Language Roundtable (ILR) scale is a set of descriptions of 
abilities to communicate in a language. It was originally developed by the United States 
Foreign Service Institute, and is still widely known as the FSI scale. It consists of 
descriptions of five levels of language proficiency. 

Levels 

Elementary proficiency L-l 

Definition 

Sufficient comprehension to understand utterances about basic survival needs and 
minimum courtesy and travel requirements in areas of immediate need or on very 
familiar topics, can understand simple questions and answers, simple statements and very 
simple face-to-face conversations in a standard dialect. These must often be delivered 
more clearly than normal at a rate slower than normal with frequent repetitions or 
paraphrase (that is, by a native used to dealing with foreigners). Once learned, these 
sentences can be varied for similar level vocabulary and grammar and still be understood. 
In the majority of utterances, misunderstandings arise due to overlooked or 
misunderstood syntax and other grammatical clues. Comprehension vocabulary 
inadequate to understand anything but the most elementary needs. Strong interference 
from the candidate's native language occurs. Little precision in the information 
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understood owing to the tentative state of passive grammar and lack of vocabulary. 
Comprehension areas include basic needs such as: meals, lodging, transportation, time 
and simple directions (including both route instructions and orders from customs 
officials, policemen, etc.)- Understands main ideas. 
Limited working proficiency L-2 
Definition 

Sufficient comprehension to understand conversations on routine social demands 
and limited job requirements. Able to understand face-to-face speech in a standard 
dialect, delivered at a normal rate with some repetition and rewording, by a native 
speaker not used to dealing with foreigners, about everyday topics, common personal and 
family news, well-known current events and routine office matters through descriptions 
and narration about current, past and future events; can follow essential points of 
discussion or speech at an elementary level on topics in his/her special professional field. 
Only understands occasional words and phrases of statements made in unfavorable 
conditions, for example through loudspeakers outdoors. Understands factual content. 
Native language causes less interference in listening comprehension. Able to understand 
facts, i.e., the lines but not between or beyond the lines. 
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General professional proficiency L-3 

Definition 

Able to understand the essentials of all speech in a standard dialect including 
technical discussions within a special field. Has effective understanding of face-to-face 
speech, delivered with normal clarity and speed in a standard dialect on general topics 
and areas of special interest; understands hypothesizing and supported opinions. Has 
broad enough vocabulary that rarely has to ask for paraphrasing or explanation. Can 
follow accurately the essentials of conversations between educated native speakers, 
reasonably clear telephone calls, radio broadcasts, news stories similar to wire service 
reports, oral reports, some oral technical reports and public addresses on non-technical 
subjects; can understand without difficulty all forms of standard speech concerning a 
special professional field. Does not understand native speakers if they speak very quickly 
or use some slang or dialect. Can often detect emotional overtones. Can understand 
implications. 

Advanced professional proficiency L-4 

Definition 

Able to understand all forms and styles of speech pertinent to professional needs. 
Able to understand fully all speech with extensive and precise vocabulary, subtleties and 
nuances in all standard dialects on any subject relevant to professional needs within the 
range of his/her experience, including social conversations; all intelligible broadcasts and 
telephone calls; and many kinds of technical discussions and discourse. Understands 
language specifically tailored (including persuasion, representation, counseling and 
negotiating) to different audiences. Able to understand the essentials of speech in some 
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non-standard dialects. Has difficulty in understanding extreme dialect and slang, also in 
understanding speech in unfavorable conditions, for example through bad loudspeakers 
outdoors. Can discern relationships among sophisticated listening materials in the context 
of broad experience. Can follow unpredictable turns of thought readily, for example, in 
informal and formal speeches covering editorial, conjectural and literary material in any 
subject matter directed to the general listener. 
Functionally native proficiency L-5 
Definition 

Comprehension equivalent to that of the well-educated native listener. Able to 
understand fully all forms and styles of speech intelligible to the well-educated native 
listener, including a number of regional and illiterate dialects, highly colloquial speech 
and conversations and discourse distorted by marked interference from other noise. Able 
to understand how natives think as they create discourse. Able to understand extremely 
difficult and abstract speech. 
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APPENDIX 2: Data Collection Sheets (DCS) 

Main Study 

Weekly Data Collection Sheet - Student 

Name: Date: □ I heard this audio 

before. 

Week of basic course: 1st / 2nd session for the week (circle); 

Transcription area for audio type (circle distortion type): Phone / Static / Babble 



2 


3 


4 


5 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 



Feedback: 

Please check: (l=low, 5=very high) 1 

Difficulty □ 

Stress □ 

Confidence in your transcription accuracy □ 

Please add verbal input regarding the above parameters, for example: Phone Line 
is difficult, but not stressful, while the other Babble stresses me out. When I am more 
confident I stress out less. 

I stress out more with phone line / Babble. What do you find challenging / 
helpful? What would make it a more useful learning 

experience? 



Teacher only: #words %Coherency: #Errors: #Prom. Err. Type_ 
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Weekly Data-Collection Sheet - Student (conf) 

Please circle level of agreement with the statements for TODAY'S activity , and optionally 
add beside each one: your feedback, for example: (a) any explanatory comments / more or less 



stressing / it makes me understand that. . .. / it develops my hearing / reading / it connects to 
the skills .... Etc. Please continue on the back of the page if you need more space. 



Metacognition - It helped me understand the process of what is 
happening during transcription. 


No / Some / Yes 


Vocabulary - 1 feel better when I know the words I transcribe. 


No / Some / Yes 


Clip sounds better if I do repeated listening (listen 2-4 times to the 
whole clip before starting transcription). 


No / Some / Yes 


It is helpful to bridge the gap over unknown word, plug-in a synonym 
and then "to get" the real word. 


No / Some / Yes 


It is helpful to put-aside challenging words and get to them later. 


No / Some / Yes 


It is helpful when I understand the theme and can map-out key- words. 


No / Some / Yes 


It helps me to Employ Grammar Rules: 




Note the noun, find the agreement. 


No / Some / Yes 


Note the verb, find the tense / possession / number / gender. 


No / some / Yes 


Note the adjective, find the number / gender. 


No / Some / Yes 


It helps me to identify and use the language characteristics when I 
transcribe. 




- Writing characteristics. 


No / Some / Yes 


Speaking Characteristic (speed, running words together. . .). 


No / Some / Yes 


Pronunciation characteristics "n' . . .). 


No / Some / Yes 
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It is helpful to write down an unfamiliar word in the best way I can, 
and then to 


No / Some / Yes 


It was helpful to defer my listening to challenging words, go to fill-out 
this feedback and go back to the listening. 


No / Some / Yes 


After listening to a previous audio while reading its transcription it 
was easier to transcribe this text today. 


No / Some / Yes 


After listening to a previous audio while reading its transcription 
AND saying it out-loud it was easier to transcribe this text today. 


No / Some / Yes 


After voweling other texts using the recording it was easier to 
transcribe this text today. 


No / Some / Yes 


After using "Transcription puzzle" with other audios it was easier to 
transcribe another text today. 


No / Some / Yes 


Tell about your best transcription technique, approach: 
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Weekly Data Collection Sheet - Student (conf) 
Name: Date: 

Today I transcribed (circle / check): □ Phone □ Static □ Babble 
Comparative feedback 

Listen to yours and 2 other distortions only . After listening to all distortions, answer: 
Which distortion in your opinion was hardest TODAY ? □ 2Phone DStatic □ Babble 

Can you point what was harder? Explain / estimate why? Any other thought? 



Ask your teacher for the solution page 

Listen to / read the Solution and / or the Original clip (your choice, whatever 
you're most comfortable with), and gist in English 



Write down the Hebrew words that you didn't know in 

today's distortion, while first transcribing (on the previous feedback pages). 
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1st -phase Final Feedback 

Main Study Final Feedback 

Thank you for participating in this feedback session. I would like to ask you some 
questions about your experience with the special listening transcription exercises that we 
have done over the past six transcription sessions, in which you transcribed three types of 
distortions. Before we move to the next phase, please help summarize the past three 
weeks and answer the questions: 

a. Please compare the level of stress that you experienced in transcription 
exercises at the beginning vs. the end of the study. 

b. How difficult was each type of distortion for you? Did this change during 
the sessions? 

c. What were the most helpful teaching / listening techniques that were used 
during the study? What made each one helpful? The list of the techniques is 
on the board, on the end of this document and is available as a hard copy 
with the teacher, so you can look at to refresh your memory if you need to. 

d. Would you recommend that future students go through the same kind of 
instructional experience? Why? 

e. What are some "tips" you can give a future student to better understand 
distorted audio? 



f. 



Is there anything I haven't asked about that you would like to add? 
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Weekly Data Collection Sheet (reference list for student's feedback) 

Please circle level of agreement with the statements for TODAY'S activity , and 
optionally add beside each one: your feedback, for example: (a) any explanatory 
comments / more or less stressing / it makes me understand that. . . . / it develops my 
hearing / reading / it connects to the skills .... Etc. Please continue on the back of the 
page if you need more space. 



Metacognition - It helped me understand the process of what is 
happening during transcription. 


No / Some / Yes 


Vocabulary - 1 feel better when I know the words I transcribe. 


No / Some / Yes 


Clip sounds better if I do repeated listening (listen 2-4 times to the 
whole clip before starting transcription). 


No / Some / Yes 


It is helpful to bridge the gap over unknown word, plug-in a synonym 
and then "to get" the real word. 


No / Some / Yes 


It is helpful to put-aside challenging words and get to them later. 


No / Some / Yes 


It is helpful when I understand the theme and can map-out key- words. 


No / Some / Yes 


It helps me to Employ Grammar Rules: 




Note the noun, find the agreement. 


No / Some / Yes 


Note the verb, find the tense / possession / number / gender. 


No / Some / Yes 


Note the adjective, find the number / gender. 


No / Some / Yes 


It helps me to identify and use the language characteristics when I 
transcribe. 




Writing characteristics. 


No / Some / Yes 


Speaking Characteristic (speed, running words together. . .). 


No / Some / Yes 


Pronunciation characteristics "n' . . .). 


No / Some / Yes 
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It is helpful to write down an unfamiliar word in the best way I can, 
and then to 


No / Some / Yes 


It was helpful to defer my listening to challenging words, go to fill-out 
this feedback and go back to the listening. 


No / Some / Yes 


After listening to a previous audio while reading its transcription it 
was easier to transcribe this text today. 


No / Some / Yes 


After listening to a previous audio while reading its transcription 
AND saying it out-loud it was easier to transcribe this text today. 


No / Some / Yes 


After voweling other texts using the recording it was easier to 
transcribe this text today. 


No / Some / Yes 


After using "Transcription puzzle" with other audios it was easier to 
transcribe another text today. 


No / Some / Yes 
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Advanced Study 

Weekly data-collection sheet - student 

Name: Date: □ I heard this audio before. 

Week of basic course: 1 st / 2 nd session for the week (circle); 

Audio type: Muffled 

BEFORE today's transcription , please warm-up in any way you see fit. Please provide a 
report and feedback (electronically) of your warm-up technique . Available are all 

previous materials. 

Transcription area (on the hard-copy paper): 



Feedback : 

Please check: (l=low, 5=very high) 



Confidence in your transcription accuracy □ 



I 


2 


3 


4 


5 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


L 



Please submit feedback (electronically) 

Please add verbal input regarding your experience, along these lines: 

- Difficulty: How difficult is TODAY'S clip? Compare/connect to any other experience 
you had. 

- How efficient was the warm up? 

- What do you find challenging / helpful? 

- What would make it a more useful learning experience? 

- What techniques did you employ today? To what challenge did the help? 

- ANY other input that comes to mind is welcome! 

Ask your teacher for the solution page. 

Gist in English, and write IN HEBREW the words that you didn't know. 



Teacher only: #words %Coherency: 
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Advanced-phase final Feedback 

ANONYMOUS 

- Please save this file under a random name that will not identify you. 

- You do not have to answer any of it, if you don't wish to do so. 

Thank you for participating in this feedback session. I would like to ask you some 
questions about your experience with the special listening transcription exercises that we 
have done over the past nine transcription sessions (competing speech and muffled), in 
which you transcribed two types of distortions. 

Before we move to the next phase, please help summarize the past four week and answer 
some or all of the questions, as you wish: 

1. Estimate how this activity helped you and added to your language skills. 



2. Would you recommend that future students go through the same kind of 
instructional experience? Why? 



3. What are some "tips" you can give a future student to better understand distorted 
audio? 
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4. If you want this activity to continue, how and in what form do you prefer it to be? 

Here are some suggestions, you can mark some of them (BOLD THE LINES), or 

add your own ideas (UNDERLINE IT). 
Remember: NONE of these activities will have any effect on your grades as it will be 
part of the research. 

a. I prefer to stop and forget from this research, continue in the regular 
curriculum. 

b. Continue in the same format (twice a week, all feedbacks as usual). 

c. Continue in a shortened format (different distortions, twice a week, 
NO typed-in feedback, just the regular hand-out (transcription, gist & 
the difficulty-stress-confidence evaluation). 

d. Same as "c", only once a week. 

e. A "weekly challenging transcription hour" of authentic materials in a 
super-hard level (level I taught until now 15-20 weeks ahead of the 
week you are at), with tailored instruction (such as puzzles and other 
instruments and assignments that you will be able to do that will 
simplify it instruction) with the regular hand-out. 

f. Your own idea / suggestion: 



5. Is there anything I haven't asked about that you would like to add? 
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Weekly Data Collection Sheet - Teacher (optional) 

Teacher notes / "mini interviews" with students during transcription, such as 
student remarks, suggestions for improvement, special requests / questions, 
demonstration of stress / relief, pattern of responses / complaints. 



Date: Week: 1 st / 2 nd class session for the week (circle) 

Student name: 



Student name: 



Student name: 
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APPENDIX 3: Technical Procedures and Audio-Clip Preparation 

Audio distortions were applied using the Adobe® Audition® 3 software program. 
The program is designed to process audio and was used for the purpose of this research. 
Distortions were designed to simulate real-life situations. 
Main Study 

Normalization of Parent Clip 



All original (parent) clips were normalized to have similar amplitude levels. The 
following procedure was applied to the parent clip: 
• Step 1: Open the parent clip (Figure 34). 




Figure 34. A graphic representation of an audio excerpt. 



• Step 2: Amplify the clip so most of its peaks will be cut (Figure 35). Always check 
that a distortion (saturation) does not occur when hearing the highest-amplitude areas. 
Then run the command: Hard Limit to (-O.ldb) as shown in Figure 35. 
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Figure 35. Audio excerpt amplified to cut most of its peaks + "hard limit" command. 

• Step 3: Attenuate the clip to (-15db) to normalize it to a standard (Figure 36). The 
clip is now set and ready for further process, adding Competing Speech, Babble or 
phone line filtering. This setting will be referred to as "Normalized Clip." 
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Phone-Line Distortion 

Phone distortion 1 

The first phone line simulation used in Sessions 1-3 was obtained using the 
"Normalized Clip" and running it through "Telephone Voice" function, offered by the 
Adobe Audition program's menu. 

• Menu path: Favorites /Telephone Voice (Figure 37). Compare with Figure 36 to see 
the difference in waveform. 



|- Mai 
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Figure 37. "Phone-Line 1" distortion 
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Amplify as needed to receive the final "Phone 1" distortion (Figure 38). 




Figure 38. Final Phone-Line 1 distortion. 
Phone distortion 2 

The second phone line simulation was used in Sessions 4-6. This distortion was 
processed using the "Normalized clip" and the following functions: 
• Command: Effects/Delay and Echo/Dynamic Delay (Figure 39). 




■alfrallfrafta 



Figure 39. Normalized clip and the process command to "Phone Line 2" distortion. 
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• The program presents the screen that allows the user to choose different modulations. 
Choose "Slow Loop Delay" modulation (Figure 40). 
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Figure 40. Application of the "Slow Loop Delay" modulation. 

• Amplify as needed to receive the final "Phone 2" distortion (Figure 41). 
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Figure 41. Final "Phone 2" distortion. 
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Pink-Noise ("Static") Distortion 

Pink Noise Source 

Pink Noise was generated by the Adobe® Audition® 3 program, using the "Pink 
Noise" function offered by the Adobe Audition program. Menu path: Generate/Noise. . . 
and then choose the appropriate Radio Controls. The amplitude level of the pink noise 
was set to (-18db). 
Static process 

• "Normalized Clip" with an amplitude of (-15db), added Static with an Amplitude 
of (-18db). 

• Bring the two clips into two windows in "Multi track" mode (Figure 42). 



jfSfiginal - Rodney Rodr {Sessi non #1) 




Figure 42. Audio excerpt vs. Pink Noise waveform. 

Main clip with (-15db) amplitude (top), added Static of (-18db) level. 
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• Perform File/export/ AudioMixDown function in order to combine the two clips 
(Figure 43). 
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Figure 43. The combining of two clips on top of each other. 



• Amplify the clip to bring: noise portion to (-12db) level and main audio to (-9db), as 
shown in Figure 44. 




Figure 44. Final "Static" clip of main study. 
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The same protocol of adding noise was used throughout the preparation of all 
"static" clips, as well on the "Babble Noise" distortion clips, in order to maintain a 
similar signal-to-noise ratio. 

Considerations for the Energy-Mask Noise Distortion 

The chosen signal used for the energy-mask interference is a pink-noise signal. 
While white noise was considered as well, PN is a better choice over white noise because 
its spectral structure resembles the static interference of typical communication systems 
more than a white-noise signal does. Although the amplitude of all frequencies in white 
noise is equal, in PN and communication systems the amplitude of lower frequencies is 
higher than the amplitude of higher frequencies. This difference equates the energy 
invested in the lower range of the spectrum to that invested in the higher range and is 
more adaptable to the human voice range of 0-4000 Hz. Figure 45 demonstrates the 
waveforms of white noise, pink noise, aircraft communication system static, and 
Amplitude Modulation (AM) radio static (all without human voice). 




Figure 45. Waveforms of White Noise, Pink Noise, and two typical communication 
systems' static. 
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Figure 45 shows that the amplitude composition of the Pink Noise is less 
homogenous and more similar to the communication system static than the white noise. 
This observation is strengthened by the spectral analysis of the same signals, as shown on 
Figure 46. It shows that the white noise color (representing spectral structure — 
amplitudes of signals at each frequency) is uniform throughout all frequencies, while the 
spectral expression of Pink Noise and communication static show a bigger spectral 
similarity (brighter colors in the lower frequencies). 




Figure 46. Spectral Analysis of White noise, Pink noise, and Static in two typical 
communication systems. 



The spectral vs. amplitude composition of static resembles the pink noise more than the 
white noise. 

Babble-Noise Distortion 

Babble noise was chosen to represent human interference typical of noisy 
environments. It was added to audio clips using the same procedures and standards of the 
Pink-Noise process described above. Babble-Noise was downloaded from 
http://spib.rice.edu/spib/select noise.html . 
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Advanced Study 
Basic Clip 

The "Basic Clip" procedure is the first standard process that all original clips went 
through. After this process, each clip was processed according to the specific distortion 
protocol, which produced one of the three options: a competing speech clip, a Muffled 
distortion clip or a Chopped distortion clip. 

The basic clip procedure is processed using phone-line noise and two procedures 
described in the Main Study portion of this appendix: amplitude standardization and 
"static" process, as follows: 

• The original "Parent" clip goes through a phone line process, by using the 
Favorites/Telephone Voice of the program's menu. 

• Amplitude standardization of the clip as described in Appendix 3: "Main Study 
/Normalization of Parent Clip, Figure 34, Figure 35 and Figure 36. 

• Perform Pink-Noise ("Static") distortion as described in Figure 42, Figure 43 and 
Figure 44, which should bring the Static background noise to (-12db) and the main 
message level to be 3db higher: A (-9db) amplitude level (Figure 44). 

The result should be an original clip that was processed to standardize its 
amplitude of the main message to (-9db) and its background static to (-12db). This clip 
type was the starting point for all other Advanced Study clips. 
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Competing Speech Clip 

This distortion comprises two parts: (a) the main message clip that is the required 
text to transcribe, processed as described in the "Advanced Study/Basic Clip" portion of 
this appendix and (b) a clip that was used as the overlapping speech, which will compete 
with the main message. This is part of a news report in the Hebrew language. These two 
clips were merged together to create the final "Competing speech" clip. 
Preparing "Basic clip" -the main audio part of the clip 

This clip is prepared according to the procedure described in "Advanced 
Study'V'Basic Clip" in this appendix. The procedure results in a "Basic clip" with 
(-12db) static level and (-9db) main message level (Figure 44). 
Preparing the "overlapping speech" part of the clip 

The overlapping Competing speech was taken from the Israeli News Broadcast 
(IBA), the weather forecast. The clip was normalized to (-12db) level. The following 
procedure was applied on the IBA weather clip: 
• Step 1: Open the IBA weather clip (Figure 47). 




Figure 47. A graphic representation of the overlapping original competing speech audio. 
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• Step 2: Amplify the clip so most of its peaks will be cut as shown on Figure 35. 
Always check that a distortion (saturation) doesn't occur when hearing the highest- 
amplitude areas. Then run the command: Hard Limit to (-O.ldb) as shown in Figure 
35. 

• Step 3: Attenuate the clip to (-12db) to normalize it to a standard of (-12db) level 
(Figure 48). The clip is now ready to be merged with the clip carrying the main 



message for transcription. 




Figure 48. Normalized Clip for overlapping competing speech. 
Merging "Basic Clip " with overlapping speech clip 

• Perform File/export/AudioMixDown function, as shown in Figure 43, on the "Basic 
clip" and the "Competing speech" in order to combine the two clips together as 
shown in (Figure 49). 
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Figure 49. The "Main Clip" and the Overlapping clip that produce the "Competing 
Speech" clip, before the merger. 



Note that the main clip carried a (-9db) level information for transcription, as well 
as a (-12db) static background, while the competing speech had an amplitude level of 
(-12db), the same level as the static background. The final result was a clip with a (-9db) 
main message and two overlapping (-12db) signals that aimed to distort and mask the 



main level (Figure 50). 




Figure 50. The product of the "Main Clip" and the Overlapping clip after the merger. 
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• The clip (Figure 50) was amplified to bring the background noise (composed of static 
and weather overlapping noise) to (-9db) db, as shown in Figure 51: 




Figure 51. The final "Competing Speech" clip. 
"Muffled" clip 



This clip is produced by running a "Basic Clip" through a protocol of muffling 
distortion as follows: 

Preparing the "Basic clip" — the main part of the audio clip 

This clip is prepared according to the procedure described in "Advanced 
Study'V'Basic Clip" in this appendix. The procedure results in a "Basic clip" with 
(-12db) static level and (-9db) main message level (Figure 44). 
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Performing the muffling effect 

• Attenuate the "Basic" clip to bring the static level to (-24db) in order to prevent 
saturation when applying equalizer (Figure 52). 




Figure 52. Attenuated "Basic" clip before equalizer procedure. 
• Apply the muffling effect using equalizer function (Figure 53). 
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Figure 53. Applying equalizer function. 



197 



Set equalizer as shown in Figure 54. 
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Figure 54. Equalizer setting. 
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Figure 55. Final "Muffled Clip. 
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"Chopped" clip 

This clip is produced by running the "Basic Clip" through a protocol of 
Chopping distortion. 

Preparing the "Basic clip "-the main part of the audio clip 

This clip is prepared according to the procedure described in "Advanced 
Study'V'Basic Clip" in this appendix. The procedure results in a "Basic clip" that has a 
(-12db) static level and a (-9db) main message level (Figure 44). 
Performing the chopping effect 

• As shown in Effects view, in Figure 56, use Effects/ Amplitude and compression/ 
dynamic processing. Then create a new point at (-50db):(-50db) and drag to point of 
(-25: -95) and press "OK." 
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Figure 56. Attenuated "Basic" clip before equalizer procedure. 
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• Amplify the result to bring most peaks to around (-6db) as shown in Figure 57. 
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APPENDIX 4: Obtaining Participants' Informed Consent 

Participation in the study was voluntary. Volunteers were recruited from the 
students in the Hebrew Basic Program, DLI class 21501HE00113 (main study 
participants) and class 21501HE00412 (preliminary study participants), by a neutral 
party, a person who plays no role in teaching them, grading them, or determining their 
status as students. In administering the informed consent form (see below), this person 
explained to the potential volunteers (a) the content of the form, including the purpose of 
the study; (b) what participation entailed in terms of time and activities; (c) the benefits 
of participation; and (d) that participation was entirely voluntary, with no penalty for 
withdrawing from the study at any time. 

The plan for any student who was not willing to participate was to transcribe the 
original non-distorted audio materials during class but not to fill out data collection 
sheets. However, all students agreed to participate and signed the consent forms. 
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Participant Informed Consent Form (preliminary pilot study) 

Dear student, 

You are invited to participate in a research study conducted by one of your teachers, Mr. 
Doron Friedman, who is working on a doctorate in the field of Education at Argosy 
University. For his doctoral dissertation, Mr. Friedman is conducting a research study 
entitled The Perception of Instructional Effectiveness, Stress, and Confidence in 
Transcription Accuracy Reported by Adults in their Early Stages of Learning Hebrew as a 
Foreign Language, During Exposure to Imperfect Audio Material. The study will last for 16 
weeks, during which the participants will participate in an experimental instructional 
program designed to help them improve in their ability to understand and transcribe audio 
passages which are in some way or ways inferior in audio quality, like those that they are 
likely to encounter in the future on the job as military linguists. Instead of the good-quality 
audio that participants are accustomed to listen to in the Basic Program, participants will 
listen to and transcribe a version of the same materials that will have been altered by the 
addition of background noise or by the use of a phone-line filter. Participants will be given 
tailored teaching and will be taught using specific teaching techniques designed to enable 
them to learn to cope with the listening challenge. During the instructional program students 
will report their experience regarding "what works" in the program, in order to provide the 
teacher with information as to how to adjust his teaching in the following transcription 
sessions. The expected result is that students will become more proficient in listening to 
audio that is less than optimal in quality. 



Student's initials: 
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You are invited to participate in the preliminary pilot study. It is identical but will 
take only 1-3 sessions. Your participation is completely voluntary and no harm will come 
to you whether you do or do not participate. If you decide not to participate, you will 
transcribe the standard listening material used in the Basic Program and will not be asked 
to provide feedback. If you do decide to participate, your transcriptions will be scored, 
but the scores will have no effect on your grades, GPA, or status in the Basic Course. 
The following rules apply to all participants: 

You are free to decline to participate, or to withdraw from the study at any 
time after you have begun. 

In providing feedback on your experience during the study, you do not have to 
answer any question or item if you choose not to, and by doing so you will 
suffer no harm. 

During each instructional session in the study you will fill out a data- 
collection sheet that will include your transcription and your feedback 
experience. 

At the end of the research, you will be interviewed by the researcher and will 
be asked to summarize your overall impression of the program. 
Security: no Personally Identifying Information (PII) will be collected. 
Transcripts will include only a target-language nickname (not participants' 
real names). These nicknames will not be reported, and will be used for 
tailored teaching purposes only. The researcher will report information from 
the study as group data only. 

Student's initials: 
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If you have any questions about the study at any time, you may ask the researcher 
in person or contact him by phone at 858-663-21 17, or by e-mail at 
doronf65 @ gmail.com 
If you agree to participate in the study, please sign below: 

I agree to participate in the above-described research. 

Date: Print Name: 



Signature 
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Participant Informed Consent Form (Main & Advanced study) 

Dear student, 

You are invited to participate in a research study conducted by one of your teachers, Mr. 
Doron Friedman, who is working on a doctorate in the field of Education at Argosy 
University. For his doctoral dissertation, Mr. Friedman is conducting a research study 
entitled The Perception of Instructional Effectiveness, Stress, and Confidence in 
Transcription Accuracy Reported by Adults in their Early Stages of Learning Hebrew as a 
Foreign Language, During Exposure to Imperfect Audio Material. The study will last for 16 
weeks, during which the participants will participate in an experimental instructional 
program designed to help them improve in their ability to understand and transcribe audio 
passages which are in some way or ways inferior in audio quality, like those that they are 
likely to encounter in the future on the job as military linguists. Instead of the good-quality 
audio that participants are accustomed to listen to in the Basic Program, participants will 
listen to and transcribe a version of the same materials that will have been altered by the 
addition of background noise or by the use of a phone-line filter. Participants will be given 
tailored teaching and will be taught using specific teaching techniques designed to enable 
them to learn to cope with the listening challenge. During the instructional program students 
will report their experience regarding "what works" in the program, in order to provide the 
teacher with information as to how to adjust his teaching in the following transcription 
sessions. The expected result is that students will become more proficient in listening to 
audio that is less than optimal in quality. 

Student's initials: 
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You are invited to participate in the study. Your participation is completely 
voluntary and no harm will come to you whether you do or do not participate. If you 
decide not to participate, you will transcribe the standard listening material used in the 
Basic Program and will not be asked to provide feedback. If you do decide to participate, 
your transcriptions will be scored, but the scores will have no effect on your grades, 
GPA, or status in the Basic Course. 

The following rules apply to all participants: 

You are free to decline to participate, or to withdraw from the study at any 
time after you have begun. 

In providing feedback on your experience during the study, you do not have to 
answer any question or item if you choose not to, and by doing so you will 
suffer no harm. 

During each instructional session in the study you will fill out a data- 
collection sheet that will include your transcription and your feedback 
experience. 

At the end of the research, you will be interviewed by the researcher and will 
be asked to summarize your overall impression of the program. 
Security: no Personally Identifying Information (PII) will be collected. 
Transcripts will include only a target-language nickname (not participants' 
real names). These nicknames will not be reported, and will be used for 
tailored teaching purposes only. The researcher will report information from 
the study as group data only. 

Student's initials: 
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If you have any questions about the study at any time, you may ask the researcher 
in person or contact him by phone at 858-663-21 17, or by e-mail at 
doronf65 @ gmail.com 
If you agree to participate in the study, please sign below: 

I agree to participate in the above-described research. 

Date: Print Name: 



Signature 
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APPENDIX 5: Description of Research Progress and Class Procedures 
Administration of consent forms 

1. Invited Dr. Gordon Jackson, an uninvolved person, to present an introduction 
to the research. 

2. Entered the class to explain the research in details + a presentation and 
examples of clips. It was emphasized that no harm would come to anyone 
who does not wish to consent and that a student who does not participate will 
transcribe the regular curricular material. 

3. Left the class - while Dr. Jackson disseminated the consent forms (Appendix 
4). All students agreed to participate. 

Presentation for students 

(Slide 1) 

Problem 

- Students have difficulty understanding imperfect audio (background noise, 
phone line) although in most cases - the transcript is relatively simple. 
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(Slide 2) 
Solution 

Develop distorted audio (from current curricular materials) 

No need for validation & verification process for students' level 

Availability of material in all levels 

Availability in all languages 

Develop (and use) teaching methods 

Evaluation of results: 

Questionnaires 

Grading of transcripts 

(Slide 3) 

Title (tentative) 

"The perception of instructional effectiveness, stress and confidence in 
transcription accuracy, reported by Adults in their early stages of learning Hebrew 
as a foreign language, during exposure to Imperfect audio material" 

What is checked: 

- Teaching methods effectiveness 

-Students' stress 

-Students' confidence 

How is it checked : 

-Qualitatively: Questionnaires, interviews. 
-Quantitatively: Accuracy check (of transcriptions) 
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(Slide 4) 

Data collection process 

Length: 16 weeks 

Extensiveness: 2 hours a week (per student) 
-Preliminary pilot study 



(Slide 5) 

Demonstration of clips + audio clips examples. 




(Slide 6) 

Teaching methods 
Vocabulary acquisition 

Gap Bridging 

Find a synonym 

Put- Aside words 

Keyword Mapping, Find the Theme 

Read- Listen-Speak 

Employ Grammar Rules 

Use of language characteristics 

Write down the word to the best of your ability 
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Slide 6 (continue) 

Punctuate the text using audio recording 
Methods designed for high-challenge/low-quality audio 

Transcription puzzle 
Text- Accompanied Audio 
Deferred Listening 

Connecting student's challenge with instructional solution 
(Slide 7) 

Data Collection Sheet. 

WeekjV Data-Collection Sheet - Student 

Name: Date: C I heard this audio before. 

Week: 1 3 V 2 nd session for the week (circle), Today's distortion type 

Transcription area for audio t}pe (circle): Phone ■ Static I Babble 



Feedback 

Please check: low average high 

Difficulty: C □ □ 0 □ 

Stress: □ D □ D □ 

Confidencein your transcription accuracy: C C C C C 

Please add verbal input regarding the above parameters, for example: Phone Lineis difficult, but 
not stressful, while the other Babble stresses me out. When I am more confident I stress out 
lessT stress more with phone line ' Babble. Wlrat do you find challenging helpful' 1 What w-ould 
make it a more useful learning experience? 
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Main Study Log 

Instruction and transcriptions, sessions 1-6, Phone Line / Static / Babble 

Research started on the 13 th week of the course, of 47 total weeks. Date: Feb-1 1-2013 
Before class 

Creating a structural method for clip / distortions assignment, by Preparation 
of the session's clips: Phone line is clip #1; Static is clip #2; Babble is clip #3. 

- Prepare DCSs. 
In Class 

Assign each student with a permanent number (1, 2 or 3), which will be the 

student's number for all future sessions (of Main Study only). 

Teach where to find the audios + creating a shortcut + students copy their 

audios to their desktops. Each student will transcribe the clip which carries 

the number of the student himself (1, 2 or 3). 

Dissemination of Data collection sheets (DCS) part (a). 

Explain the page, especially feedback part of DCS (a). 

Transcript + feedback. Allow reasonable time for all (10-15 min). 

- Collecting DCS(a). 

Dissemination of Data collection sheets (DCS) part (b). 

- Students fill out DCS(b) 
After class 

1. Process of questionnaires, Main errors and findings: 

Most students found Static the hardest. However, quality of transcription 
didn't show any trend (static wasn't necessarily lower performance). 



Most students said they could hear through the noise. 
Segmentation - typical error.. 

Unfamiliar words and grammatical structures were the main focus of errors. 
Prepositions not recognized (again - SS not sure of the correct preposition) 
Decisions: 

Action: reduce the level of SNR (increase Statis and Babble) 
Give fewer words in transcript, (reduced from 44 to 34) 
Focus on decoding unfamiliar words. 

Focus on techniques to write down in the best way that will convey the 
unfamiliar word as close to how it is heard. 

Students need to practice on correctly choose the Hebrew letters in order to 
convey the right sound (letters and voweling!). Will assign HW. 
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Session 2 

In Class 

1 . Ask students to give an example of techniques from own experience, and 
discussion. 

2. Explain main errors + HW. 

3. Start transcription: 8:15 (worked perfectly) 

After class 

1 . Process of questionnaires 

2. . , Main errors and findings: 

I developed a more quantitative approach to assess coherency. 

WENT BACK TO 1st SESSION and refined the transcription evaluations 

accordingly. 

SNR change helped and separation between students and distortions shows. 

Continue with similar SNR. 

Most students think STATIC is most challenging. 

Main struggle: coping with unknown words / structures / prepositions (since they 
don't know which preposition to use, it is an "unknown" part. 
Main errors: segmentation (either running multiple words together OR breaking a 
word in multiple parts) 
Ideas / decisions: 

If babble continues to sound easier than static: consider English competing speech 
(weather forecasters?) 

Check logarithmic ratio between coherency : # of errors. 
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Assign HW: transcription voweling, letters recognition, letters sound. 
Session 3 
Before class: 

Increased level of Static and Babble. 
Class Teaching: 

Focus on write WHAT YOU HEAR and write it down to the best of ability if you 

don't understand the word. 

Note agreements, it helps expect the right word. 

- Not "nx", then expect "n" etc. 
Note crossing of letters. 

- "n/D" becomes "n". Show Amotz. 

- ciVira nnD] nriD 

Show typical errors to the students: 

Present on the Smart Board two typical transcriptions with errors. 
After class 

Process of questionnaires, Main errors and findings: 

Decline in segmentation errors. 

Students noted significant of difficulty in Static, Babble. 
Students report easy Phone line 
Ideas / decisions: 

Added another technique: repeated listening. 
Improvement of DCS(a,b) 
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Next time: In order to build on their experience, application of learning strategy: 
warm up by instructing students to listen to PREVIOUS IDENTICAL distortion 
AS Today + text. ONLY then work on today's piece 

Change the phone line - into something more disturbing - a type of electronic 
interference, (regular phone too easy according to feedback) 
Teaching next time: 

Instruct that they use the new technique. 

- Instruct to listen to PREVIOUS IDENTICAL distortion + text, ONLY then work 
on today's piece. 
Notify slight changes on DCSs 



Session 4 

Before Class: 

New standards for audio sets: 

Changed phone distortion to phone 2 (Appendix 3). 

Added NEW process for audio: Normalization of parent clip (Appendix 3). 
Noise was already amplified in Session 3. 
Teaching: 

Continue: instruct that they use the new technique. 
Go over HW (transcription, voweling, letters recognition, letters sound). 
- Instruct to listen to PREVIOUS IDENTICAL distortion + text, ONLY then work 
on today's piece. 

Continue: Notify slight changes on DCSs 
After class 

Process of questionnaires, Main errors and findings: 
Stopped counting words the students produced, as it emerged as meaningless as (a) 
most counts were around the original count of words in the text, (b) segmented and 
de-segmented words balanced set each other off so it wasn't indicative of quality of 
transcription. 

Looks like segmentation errors dropped dramatically. 
Prominent errors are counted from 3 errors and up. 
Next check procedure: 

1 st round: coherency + error types 
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2 nd round: straight to date keying into excel. 
Ideas / decisions, Finding and emerging trends: 

Assign HW like session #2: transcription voweling, letters recognition, letters 
sound. 

Fast learning curve. 

Segmentation errors dramatically dropped down. 

Equal number of students moved from current distortion to babble / static, (which 

shows they are harder). 

Remind students to use Repeated Listening. 

Do repeated listening of the day's distortion. 

Do Transcription puzzle of previous / familiar text before transcription. 
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Session 5 

Teaching 

Do repeated listening of the day's distortion. 

Do "Transcription puzzle" of previous / familiar text before transcription. 
After class 

Process of questionnaires, Main errors and findings: 

Generally, results were high, all students said the challenge was much smaller. 
Ideas / decisions: 

Consulted with experienced persons regarding realistic future distortions. 
Continue the same plan on session 6 (completion of cycle 2). 
Teaching next time: 

Warm-up with of Transcription puzzle #4. Transcribe without the solution. 
Check by the teacher - point off for every needed move of word. 
Give a fill-in the blanks. Ss need to complete another word and define its 
function and explain why (infinitive, since there is another verb). 
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Session 6 

Teaching 

Do repeated listening of the day's distortion. 
Teaching: 

Give a fill-in the blanks. Ss need to complete another word and define its function 
and explain why (infinitive, since there is another verb). Teacher goes over (ask 
students to tell their options) and explains. 
(Puzzle is on the teaching document of #6) 
If time permits: 

Warm-up with of Transcription Puzzle #4. Transcribe without the solution. 
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Advanced Study 

Instruction and transcriptions sessions 

Sessions 7-10: Competing Speech 

Session 7 

Preparation; 

Chosen clip for (all class, no groups) transcription: Competing speech with some 
amplification to bring Static to a certain fixed level (Appendix 3: Competing 
Speech). 

Prepare to explain the students of the structural change in DCSs. 

Competing speech was of a female broadcaster. 

Teaching: 

Prepare students to structure change in DCSs. Ask students: 

1 . To fill out DCS (c) 1 st -phase Final Interview (it is recommended to do that on the 
previous session - the actual last session of the 1 st phase). 

2. Introduce the new structure: 

a. Ask to give feedback for regular DCSs electronically (more convenient for 
today's students, more convenient for data collection. This produced much 
better and quality feedback). 

b. DCSs contain highlighted instructions (Appendix 4). 
After class 

Process of questionnaires, Main errors and findings: 

Transcription was quite hard. 

Range of coherency: 90-39%. Average: 56% 
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Teaching next time; 

Make available to students: All sessions, including this one (7 th session) for 
warm-up. 

Materials: original texts, originals audio of clips. 

Ask students to do whatever warm-up they wish and report it. 

Then run the transcription, and on the feedback - report the efficiency of their 

warm-up. 
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Session 8 
Preparation: 

This time competing speech was of a male broadcaster. 
Teaching: 

Made available to students: All sessions' audios and solutions, including the last 
one (7 th session) materials: Original texts, originals audio of clips (both the main 
clip and the competing speech clip). 

Ask students to do whatever warm-up they wish and report it. 
Then run the transcription, and on the feedback report the efficiency of their 
warm-up. 
After class 

Process of questionnaires, Main errors and findings: 

Results improved significantly. Coherency avg. up from 56 to 70 
Some students showed huge advancement, (see results): the same student, three 
days apart. Difficulty & stress lowered, confidence increased, coherency 
dramatically increased. 
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Class overview see results (Chapter 4): 



o Most students reported that warm-up helped, most warm-up methods were 
repeating last session combined with solution (text) and noting vocabulary 
and grammatical cues. Results increased: Mostly students who warmed-up 



Warm- 
up? 


Y 


Y 


Y 


Y 


Y 


Y 


Y 


Y 


Y 


Y 


N 


N 


N 


Change 
(points) 


47 


31 


25 


23 


18 


15 


5 


4 


-6 


47 


4 


2 


-2 



This clip was evaluated by Ziggy (OPI tester) and Esti Burstein (another teacher): 
it was compared with clip 7, to see if any of them was harder. Both determined 
that they are of the same difficulty, Esti stated that clip 8 might be just slightly 
harder than 7. Since results of 8 were higher than 7, then the higher coherency 
could not be attributed to difference in clips' difficulty. 
Ideas / decisions; 

Encourage the three that didn't warm-up to do so. Their grade changed only by 
few points (4,2,-2) 

Next time do competing speech both male and female. 
Next time include all tenses in text. 
Teaching next time: 

Warm-up same as last time. Note techniques: 

o Isolate yourself, including senses, from the environment, 
o Isolate different transmissions; know how to focus attention on a chosen 
one. 
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o Listen to transmissions from all weeks, 
o Reduce tension 
o Using the solutions: 

■ Focus on the problematic syllables, sounds etc, repeat theses out 
loud. 

■ Focus on grammar. 
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Session 9 
Preparation: 

This time competing speech was of a male broadcaster. 

This clip (#10) was evaluated by Ziggy (OPI tester) and clips 7, 8 to see if any of 
them was harder. Ziggy determined that they are same difficulty or maybe #9 is 
very slightly more difficult since competing speech changes voices half way 
through the clip. This can be balanced by the fact students could listen to previous 
clips, to cancel the surprise element. 
Teaching: 

Made available to students: All sessions' audios and solutions, including the last 
one; materials: Original texts, originals audio of clips (both the main clip and the 
competing speech clip). 

Ask students to do whatever warm-up they wish and report it. 
Then run the transcription, and on the feedback report the efficiency of their 
warm-up. 
After class 

Process of questionnaires, Main errors and findings: 

Coherency avg. up from 70 to 75: Results improved, 5 points on average. 
Potential for some improvement still exists. Some students had lower grades. 
Also, "Monday effect" might have played a part here. 
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Ideas / decisions: 

Do another run (#4 for this distortion) to see if here will be a plateau, 
In future weeks not to do a Monday session, but a Tue (after "Monday effect" 
suggested by Eyal and Paz). 
Teaching next time: 

Same as last time: warm-up 
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Session 10 
Teaching: 

Made available to students: All sessions' audios and solutions, including the last 
one; materials: Original texts, originals audio of clips (both the main clip and the 
competing speech clip). 

Ask students to do whatever warm-up they wish and report it. 

Then run the transcription, and on the feedback report the efficiency of their 

warm-up. 

Students Reporting decreasing tension & difficulty, and higher confidence. Last 4 
sessions of this distortion, described in chapter 4. 

Some students thought that the last clip was easier than others. One commented 
that the last one lacked future tense. 
Ideas / decisions: 

In the future, Record all clips at one time, and distort them together for higher 
equality among them. Have at least 2 other professional confirm all texts are 
linguistically in the same difficulty. 
Continue with 2 more distortions. 
Teaching next time: 

Since it will be a new distortion - go straight to the clip. 
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Sessions 11-15: Muffled distortion 
Session 11 

One student was removed from the class. Results now: Of 1 1 students only. 

All future sessions of next distortions were recorded / prepared at the same time 
and were: 

o Checked (Ziggy (OPI Tester), Esti T, Anat) for equality in language level. 

o Number of words for each voice in the clip was as equal as possible. 

o Recorded at the same day. 

o Were distorted together. 

o Distortion process see Appendix 3. 

Teaching: 

Since it will be a new distortion - go straight to the clip. 

Process of questionnaires, Main errors and findings: 

Clip was too easy, 9 Of most students scored over 90%. 

Ideas / decisions: 

Muffle the clip more. More rigorous muffling was decided with help of the a 
strong student who confirmed that now "he has to work hard" to pick up the 
message, call it Muffle2 

The muffle2 setting became the formal protocol for muffling. 

Teaching next time: 

Listen to previous clip, get ready for a harder one. 
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Session 12 
Teaching: 

Since it is a new distortion - go straight to the clip. 

After class: Process of questionnaires, Main errors and findings: 

Muffle2 works: Clip was challenging enough - avg. coherency: 60%. 

Ideas / decisions: 

Students indicate this research so far has helped them progress in listening and 
estimate that it helped them on their tests. They see their own progress and 
estimate that the without the research and gradual training they wouldn't have 
been able to transcribe the distortions. 

Teaching next time: 

Listen to previous clip. 
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Session 13 
Teaching: 

Listen to one of the previous clips, I suggested to listen to the previous #12 (same 
distortion) while reading its solution. 

After class, Process of questionnaires, Main errors and findings: 

Clip continued to be challenging - avg. coherency: 57%, students indicators 

(difficulty-stress-confidence) continue to show difficulty. 

Ideas / decisions: 

Continue with same distortion. 

Assign this clip with Puzzle for HW. 

One student indicated the volume wasn't high enough - provide next clip with 
elevated volume (however tried to minimize saturation distortion). 

Teaching next time: 

Listen to previous clips, do puzzles. 

Create puzzle of #12 (same muffles2 distortion) to be available for next warm-up 
/ home training. 
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Session 14 
Teaching; 

Listen to previous clips 

Puzzles of #12, 13 (same muffles2 distortion). 

After class, process of questionnaires, Main errors and findings: 

Barrier was broken, avg. coherency: 74%, student's indicators (difficulty-stress- 
confidence) show relief of stress and difficulty, elevation of confidence. 

Ideas / decisions: 

Continue with same distortion; want to see the stabilization to a plateau. 
Assign this clip with Puzzle for HW. 

One student still shows significant challenge. Will suggest to him the puzzles 
again. Check how he does on the HW (another non-related authentic phone-call: 
Aunti & Sarah) 

Teaching next time: 

Warm-up: same distortion puzzles. 
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Session 15 
Teaching: 

Listen to previous clips. 

Puzzles of #12, 13, 14 (same muffles2 distortion). 

After class, process of questionnaires, Main errors and findings: 

Avg. coherency: 79%, learning curve formed. Student's indicators (difficulty- 
stress-confidence) show relief of stress, elevation of confidence. 
Anonymous questionnaire: At the end of this session (# 5 of this distortion) 
Students were given an anonymous questionnaire to see their feedback of this 
distortion (Appendix 4). 

Feedback results: 

Students appreciate the contribution of research to their ability. No one asked to 
stop the research. 

o 4 students wanted to continue in a shortened format (different distortions, 
twice a week, NO typed- in feedback, just the regular hand-out 
(transcription, gist & the difficulty-stress-confidence evaluation). Decided 
to continue this option with another distortion for 3-4 sessions as needed. 

o 5 wanted A "weekly challenging transcription hour" of authentic materials 
in a super-hard level (level I taught until now 15-20 weeks ahead of the 
week you are at), with tailored instruction (such as puzzles and other 
instruments and assignments that you will be able to do that will simplify 
it instruction) with the regular hand-out. Decided to work on it in the 
future. 
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Ideas/decisions: 

Stop for 10 days (tests and class considerations: 2 students about to leave, 1 about 
to join, wait for the class to stabilize). 
Continue with new distortion. 



Sessions 16-18: Chopped distortion 
Teaching: 

Listen to previous clips (starting section 17) 

After class Process of questionnaires, Main errors and findings: 

Avg. coherency: section 16: 63%; 17: 89% ; 18: 87% 

Students' indicators stabilized on low difficulty and stress, high confidence. 



